Method and device for training intelligent model

Abstract

The present application is about a method and device for training intelligent models, belonging to the field of computer science. The method includes: obtaining a first network set and a second network set from a first neural network; training each neural network in the first network set based on a first sample set and a second sample set to obtain a first model set; training each neural network in the second network set based on the first sample set, the second sample set and the first model set to obtain a second model set, wherein the second model set includes m second intelligent models obtained by training the m second neural networks.

Description

Technical Field

[0001] The present application relates to the field of computers, and in particular to a method and device for training an intelligent model. Background Art

[0002] The training samples include labeled training samples and unlabeled training samples. The labeled training samples include sample data and labels, and the unlabeled training samples include sample data. For example, assuming that the training samples are samples for training license plates, the labeled training samples include license plate images and labels, and the labels are used for the license plate positions in the license plate images, and the unlabeled training samples include license plate images.

[0003] Currently, multiple training samples can be used to train a neural network, and the multiple training samples include labeled training samples. By training the neural network with the labeled training samples, an intelligent model with superior performance can be obtained. However, when the multiple training samples include both labeled training samples and unlabeled training samples, how to train an intelligent model with the same superiority is an urgent problem to be solved. Summary of the invention

[0004] In order to solve the problems in the related art, the embodiment of the present application provides a method and device for training an intelligent model. The technical solution is as follows:

[0005] On the one hand, the present application provides a method for training an intelligent model, the method comprising:

[0006] Acquire a first network set and a second network set from a first neural network, wherein the first network set includes a first neural network and m-1 second neural networks, where m is an integer greater than 1, and the second network set includes m second neural networks, wherein the m-1 second neural networks and the m second neural networks are different subnetworks in the first neural network, and the network scale of each neural network in the first network set is greater than the network scale of each neural network in the second network set;

[0007] Based on the first sample set and the second sample set, each neural network in the first network set is trained to obtain a first model set, wherein the first sample set includes a plurality of first training samples, the first training samples include first training data and a label, and the label is used to indicate a target in the first training data, the second sample set includes a plurality of second training samples, the second training samples include second training data, and the first model set includes a first intelligent model obtained by training the first neural network and m-1 second intelligent models obtained by training the m-1 second neural networks;

[0008] Based on the first sample set, the second sample set and the first model set, each neural network in the second network set is trained to obtain a second model set, wherein the second model set includes m second intelligent models obtained by training the m second neural networks.

[0009] Optionally, the training of each neural network in the first network set based on the first sample set and the second sample set to obtain a first model set includes:

[0010] Based on the first sample set, training parameters of a convolutional layer and parameters of a fully connected layer included in a neural network to be trained, wherein the neural network to be trained is any neural network in the first network set, and parameter values ​​of the parameters of the convolutional layer after training are first parameter values, and parameter values ​​of the parameters of the fully connected layer after training are second parameter values;

[0011] Fixing the parameter value of the parameter of the convolutional layer to the first parameter value, training the parameter of the fully connected layer included in the neural network to be trained based on the first sample set and the second sample set, wherein the parameter value of the parameter of the fully connected layer after training changes from the second parameter value to a third parameter value;

[0012] The parameter values ​​of the parameters of the fully connected layer are fixed to the third parameter values, and the parameters of the convolutional layer of the neural network to be trained are trained based on the first sample set and the second sample set to obtain an intelligent model corresponding to the neural network to be trained.

[0013] Optionally, the training of parameters of a fully connected layer of the neural network to be trained based on the first sample set and the second sample set includes:

[0014] Inputting each first training sample in the first sample set into the neural network to be trained, and obtaining a first output result corresponding to each first training sample output by the neural network to be trained, and inputting each second training sample in the second sample set into the neural network to be trained, and obtaining a first output result corresponding to each second training sample;

[0015] Based on the first output result corresponding to each of the first training samples and the label included in each of the first training samples, a first loss function value is obtained through a first loss function, and based on the first output result corresponding to each of the second training samples, a second loss function value is obtained through a distance loss function, where the first loss function is a loss function other than the distance loss function;

[0016] When the fully connected layer loss function value is not a minimum value, the parameter values ​​of the parameters of the fully connected layer are adjusted based on the fully connected layer loss function value, where the fully connected layer loss function value includes the first loss function value and the second loss function value.

[0017] Optionally, the training the parameters of the convolutional layer of the neural network to be trained based on the first sample set and the second sample set includes:

[0018] Inputting each first training sample in the first sample set into the neural network to be trained, and obtaining a second output result corresponding to each first training sample output by the neural network to be trained, and inputting each second training sample in the second sample set into the neural network to be trained, and obtaining a second output result corresponding to each second training sample;

[0019] Based on the second output result corresponding to each of the first training samples and the label included in the first training sample, a third loss function value is obtained through the first loss function, and based on the second output result corresponding to each of the second training samples, a fourth loss function value is obtained through the distance loss function;

[0020] When the convolution layer loss function value is not a minimum value, the parameter values ​​of the parameters of the convolution layer are adjusted based on the convolution layer loss function value, and the convolution layer loss function value includes the third loss function value and the fourth loss function value.

[0021] Optionally, the training each neural network in the second network set based on the first sample set, the second sample set and the first model set to obtain the second model set includes:

[0022] Input each second training sample in the second sample set to each intelligent model in the first model set, and obtain a plurality of third output results corresponding to the second training sample output by each intelligent model;

[0023] Based on the multiple third output results corresponding to each second training sample, respectively obtain a label corresponding to each second training sample;

[0024] Based on the first sample set, each second training sample and a label corresponding to each second training sample, each neural network in the second network set is trained to obtain a second model set.

[0025] Optionally, the method further includes:

[0026] Based on resource information of n devices, determine n third model sets, the n devices correspond one-to-one to the n third model sets, the n devices include a first device, the third model set corresponding to the first device includes at least one second intelligent model, resources of the first device meet resources required by each second intelligent model in the at least one second intelligent model, and n is an integer greater than 1;

[0027] Based on the first intelligent model, a second intelligent model is selected for the first device from the third intelligent model corresponding to the first device.

[0028] Optionally, the selecting, based on the first intelligent model, a second intelligent model for the first device from a third intelligent model corresponding to the first device includes:

[0029] Based on the second sample set, obtaining difference information between the first intelligent model and each second intelligent model in a third model set corresponding to the first device;

[0030] Based on the difference information between the first intelligent model and each of the second intelligent models, a second intelligent model is selected from each of the second intelligent models.

[0031] Optionally, the acquiring, based on the second sample set, difference information between the first intelligent model and each second intelligent model in a third model set corresponding to the first device includes:

[0032] Input each second training sample in the second sample set into the first intelligent model, and obtain a fourth output result corresponding to each second training sample output by the first intelligent model;

[0033] Input each second training sample in the second sample set into a target intelligent model, and obtain a fifth output result corresponding to each second training sample output by the target intelligent model, where the target intelligent model is a second intelligent model in the third model set corresponding to the first device;

[0034] Based on the fourth output result and the fifth output result of each second training sample, the difference information between the first intelligent model and the target intelligent model is obtained.

[0035] Optionally, the method is applied to a central computing-edge device system architecture, the system architecture including a central device and multiple edge intelligent devices, each edge intelligent device is deployed in a different area, the computing power and storage capacity of the central device are respectively greater than the computing power and storage capacity of each edge intelligent device, the method is executed by the central device, and the second training sample in the second sample set is data collected by each edge intelligent device and received by the central device.

[0036] On the other hand, the present application provides a device for training an intelligent model, the device comprising:

[0037] An acquisition module is used to acquire a first network set and a second network set from a first neural network, wherein the first network set includes a first neural network and m-1 second neural networks, where m is an integer greater than 1, and the second network set includes m second neural networks, wherein the m-1 second neural networks and the m second neural networks are different subnetworks in the first neural network, and the network scale of each neural network in the first network set is greater than the network scale of each neural network in the second network set;

[0038] A first training module is used to train each neural network in the first network set based on a first sample set and a second sample set to obtain a first model set, wherein the first sample set includes a plurality of first training samples, the first training samples include first training data and a label, the label is used to indicate a target in the first training data, the second sample set includes a plurality of second training samples, the second training samples include second training data, and the first model set includes a first intelligent model obtained by training the first neural network and m-1 second intelligent models obtained by training the m-1 second neural networks;

[0039] The second training module is used to train each neural network in the second network set based on the first sample set, the second sample set and the first model set to obtain a second model set, wherein the second model set includes m second intelligent models obtained by training the m second neural networks.

[0040] Optionally, the first training module is used to:

[0041] Based on the first sample set, training parameters of a convolutional layer and parameters of a fully connected layer included in a neural network to be trained, wherein the neural network to be trained is any neural network in the first network set, and parameter values ​​of the parameters of the convolutional layer after training are first parameter values, and parameter values ​​of the parameters of the fully connected layer after training are second parameter values;

[0042] Fixing the parameter value of the parameter of the convolutional layer to the first parameter value, training the parameter of the fully connected layer included in the neural network to be trained based on the first sample set and the second sample set, wherein the parameter value of the parameter of the fully connected layer after training changes from the second parameter value to a third parameter value;

[0043] The parameter values ​​of the parameters of the fully connected layer are fixed to the third parameter values, and the parameters of the convolutional layer of the neural network to be trained are trained based on the first sample set and the second sample set to obtain an intelligent model corresponding to the neural network to be trained.

[0044] Optionally, the first training module is used to:

[0045] Inputting each first training sample in the first sample set into the neural network to be trained, and obtaining a first output result corresponding to each first training sample output by the neural network to be trained, and inputting each second training sample in the second sample set into the neural network to be trained, and obtaining a first output result corresponding to each second training sample;

[0046] Based on the first output result corresponding to each of the first training samples and the label included in each of the first training samples, a first loss function value is obtained through a first loss function, and based on the first output result corresponding to each of the second training samples, a second loss function value is obtained through a distance loss function, where the first loss function is a loss function other than the distance loss function;

[0047] When the fully connected layer loss function value is not a minimum value, the parameter values ​​of the parameters of the fully connected layer are adjusted based on the fully connected layer loss function value, where the fully connected layer loss function value includes the first loss function value and the second loss function value.

[0048] Optionally, the first training module is used to:

[0049] Inputting each first training sample in the first sample set into the neural network to be trained, and obtaining a second output result corresponding to each first training sample output by the neural network to be trained, and inputting each second training sample in the second sample set into the neural network to be trained, and obtaining a second output result corresponding to each second training sample;

[0050] Based on the second output result corresponding to each of the first training samples and the label included in the first training sample, a third loss function value is obtained through the first loss function, and based on the second output result corresponding to each of the second training samples, a fourth loss function value is obtained through the distance loss function;

[0051] When the convolution layer loss function value is not a minimum value, the parameter values ​​of the parameters of the convolution layer are adjusted based on the convolution layer loss function value, and the convolution layer loss function value includes the third loss function value and the fourth loss function value.

[0052] Optionally, the second training module is used to:

[0053] Input each second training sample in the second sample set to each intelligent model in the first model set, and obtain a plurality of third output results corresponding to the second training sample output by each intelligent model;

[0054] Based on the multiple third output results corresponding to each second training sample, respectively obtain a label corresponding to each second training sample;

[0055] Based on the first sample set, each second training sample and a label corresponding to each second training sample, each neural network in the second network set is trained to obtain a second model set.

[0056] Optionally, the device further comprises:

[0057] A determination module, configured to determine n third model sets based on resource information of n devices, wherein the n devices correspond one-to-one to the n third model sets, the n devices include a first device, the third model set corresponding to the first device includes at least one second intelligent model, and the resources of the first device meet the resources required by each second intelligent model in the at least one second intelligent model, and n is an integer greater than 1;

[0058] A selection module is used to select a second intelligent model for the first device from a third intelligent model corresponding to the first device based on the first intelligent model.

[0059] Optionally, the selection module is used to:

[0060] Based on the second sample set, obtaining difference information between the first intelligent model and each second intelligent model in a third model set corresponding to the first device;

[0061] Based on the difference information between the first intelligent model and each of the second intelligent models, a second intelligent model is selected from each of the second intelligent models.

[0062] Optionally, the selection module is used to:

[0063] Input each second training sample in the second sample set into the first intelligent model, and obtain a fourth output result corresponding to each second training sample output by the first intelligent model;

[0064] Input each second training sample in the second sample set into a target intelligent model, and obtain a fifth output result corresponding to each second training sample output by the target intelligent model, where the target intelligent model is a second intelligent model in the third model set corresponding to the first device;

[0065] Based on the fourth output result and the fifth output result of each second training sample, the difference information between the first intelligent model and the target intelligent model is obtained.

[0066] The technical solution provided by the embodiments of the present application may have the following beneficial effects:

[0067] The first neural network is divided into a first network set and a second network set, wherein the first network set includes the first neural network and m-1 second neural networks, and the second network set includes m second neural networks. The first training samples included in the first sample set are labeled training samples, and the second training samples included in the second sample set are unlabeled training samples. Since the scale of the neural networks in the first network set is larger than the scale of the neural networks in the second network set, even if the second training samples in the second sample set are unlabeled training samples, the first sample set and the second sample set can be used to train each neural network in the first network set, and the intelligent model corresponding to each neural network in the first network set is obtained, thereby obtaining the first model set. Then, the first model set, the first sample set and the second sample set are used to train each neural network in the second network set with a smaller network scale, so that the intelligent model corresponding to each neural network in the second network set can be trained, so that both labeled training samples and unlabeled training samples are included, and an intelligent model with superior performance is also trained.

[0068] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0070] Figure 1 It is a schematic diagram of a neural network structure provided in an embodiment of the present application;

[0071] Figure 2 is another schematic diagram of a neural network structure provided in an embodiment of the present application;

[0072] Figure 3 It is a structural diagram of a network architecture provided by an embodiment of the present application;

[0073] Figure 4 It is a flow chart of a method for training an intelligent model provided in an embodiment of the present application;

[0074] Figure 5 is a flow chart of another method for training an intelligent model provided in an embodiment of the present application;

[0075] Figure 6 is a flow chart of another method for training an intelligent model provided in an embodiment of the present application;

[0076] Figure 7 is a flow chart of another method for training an intelligent model provided in an embodiment of the present application;

[0077] Figure 8 It is a schematic diagram of the structure of a device for training an intelligent model provided in an embodiment of the present application;

[0078] Fig. 9 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0079] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0080] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0081] The first neural network is a large-scale neural network, which can be called a super network. The first neural network includes a large number of sub-networks.

[0082] The first neural network includes a large number of neurons. The neurons included in the first neural network are divided according to different depths, different widths, different convolutions and sizes, and a large number of sub-networks can be divided in the first neural network.

[0083] See also Figure 1 , divide the neurons in the first neural network into multiple different sub-networks according to different depths. Or, see Figure 2 , the neurons in the super network are divided according to different widths, and the first neural network is divided into multiple different sub-networks.

[0084] Each sub-network in the first neural network is a neural network, and the neural network is trained using training samples to obtain an intelligent model with the required function. For example, in order to be able to recognize license plates, the neural network can be trained using training samples to obtain a license plate recognition model with license plate recognition function.

[0085] For any neural network, the neural network includes a convolutional layer and a fully connected layer. The convolutional layer is used to extract features from the training sample. The fully connected layer processes the objects in the training sample based on the extracted features. For example, the fully connected layer classifies the objects in the training sample based on the extracted features.

[0086] Optionally, the convolution layer includes a feature extractor, and the fully connected layer includes multiple classifiers. The feature extractor belongs to the convolution layer of the neural network and is used to extract features from the training sample. For each classifier, the classifier classifies the target in the training sample based on the features extracted by the feature extractor.

[0087] Training samples are divided into two categories: labeled training samples and unlabeled training samples. Among them, labeled training samples include training data and labels, and the labels are used to indicate the target in the training data. For example, for a training sample used to train a license plate recognition model, the training data in the training sample is a picture including a license plate, and the label in the training sample is used to indicate the position of the license plate in the picture. The training sample is a labeled training sample. Unlabeled training samples include training data. For example, for a training sample used to train a license plate recognition model, the training sample includes a picture, and the picture includes a license plate image. The training sample is an unlabeled training sample.

[0088] The above intelligent model takes the license plate recognition model as an example, but the intelligent model can also be an intelligent model for semantic segmentation, an intelligent model for target detection, an intelligent model for key point detection and / or an intelligent model for video understanding, etc. These intelligent models can be obtained by training the neural network using training samples.

[0089] Regardless of the type of intelligent model, the classifier of the fully connected layer classifies the training data in the training sample. For example, in the license plate recognition model mentioned above, the classifier classifies each image in the training sample, and when a certain image is classified as a license plate image in the training sample, the position of the license plate image is used as the license plate position. For another example, for an intelligent model for semantic segmentation, the classifier classifies the semantics of each data content in the training sample, and classifies data content with different semantics to achieve semantic segmentation.

[0090] In an embodiment of the present application, a first sample set and a second sample set are provided, wherein the first sample set includes a plurality of first training samples, the first training samples include first training data and a label, the label is used to indicate a target included in the first training data, and the second sample set includes a plurality of second training samples, the second training samples include second training data. Wherein, each first training sample in the first sample set is a labeled training sample, and each second training sample in the second sample set is an unlabeled training sample.

[0091] Based on the first sample set and the second sample set, the first neural network and each sub-network included in the first neural network are trained to obtain an intelligent model corresponding to the first neural network and an intelligent model corresponding to each sub-network. Since the scales of the sub-networks of the first neural network are different, the scales of the intelligent model corresponding to the trained first neural network and the scales of the intelligent models corresponding to each sub-network are different, so that intelligent models of different scales can be deployed on devices with different hardware performances.

[0092] The first training samples in the first sample set and the second training samples in the second sample set may be training samples in different environments. For example, taking the above license plate recognition model as an example, each first training sample in the first sample set may be a sample obtained by photographing various license plates during the day, and each second training sample in the second sample set may be a sample obtained by photographing various license plates at night. In this way, the license plate recognition model obtained by training the first sample set and the second sample set can perform license plate recognition both during the day and at night.

[0093] See also Figure 3 An embodiment of the present application provides a network architecture, which includes a central device and multiple edge smart devices, each edge smart device communicates with the central device.

[0094] Optionally, each edge smart device and the central device are connected to a communication network, and each edge smart device establishes a network connection with the central device in the communication network to enable each edge smart device to communicate with the central device. Of course, there are other ways to enable each edge smart device to communicate with the central device, which will not be described in detail here.

[0095] The computing power and storage capacity of each edge intelligent device are much smaller than those of the central device, respectively, and the computing power and storage capacity of each edge intelligent device may be different.

[0096] The central device includes a first sample set, and each first training sample in the first sample set includes first training data and a label, where the label is used to indicate a target included in the first training data.

[0097] Optionally, the label included in each first training sample may be obtained by manually labeling each first training sample.

[0098] Each edge intelligent device is deployed in a different area. Each edge intelligent device collects data in the area where it is located, and uses the collected data as the second training sample, and sends the second training sample to the central device. Since the number of edge intelligent devices is often large, each edge intelligent device collects a large number of second training samples, so the central device will obtain a large number of second training samples. If the second training samples continue to be labeled manually, the labor cost is too high, so the central device directly uses the data collected by each edge intelligent device as the second training sample, and each second training sample is formed into a second sample set. Then the central device trains a super network based on the first sample set and the second sample set, and trains the super network into multiple intelligent models of different sizes. Each intelligent model requires different computing power and storage capacity of the device. The central device installs different intelligent models on different edge intelligent devices.

[0099] Among them, for each edge smart device, each edge smart device is deployed in a different scene. Since each edge smart device is affected by the environment in which it is located, the scene corresponding to the second training sample collected by each edge smart device is different from the scene corresponding to each first training sample in the central device.

[0100] Optionally, the central device is a central server. For example, the central device is a cloud computing server or a cloud computing server cluster. The edge smart device is a smart camera, a computer or a single-chip microcomputer.

[0101] Optionally, the network architecture is a system architecture of central computing-edge devices, for example, it can be a system architecture of cloud central computing-edge devices, that is, a central device is a server or server cluster with large-scale computing and storage capabilities (for example, the server cluster is composed of 200 graphics cards, 100 CPUs and 100T storage hard drives and other devices), and multiple edge smart devices refer to edge devices with multiple small-scale computing and storage capabilities. For example, the edge smart device is a server smaller than the server cluster (such as a computer without a graphics card or with fewer graphics cards; a single-chip microcomputer), a camera with a certain ability to carry intelligent models, or smart glasses with smaller computing power.

[0102] Figure 3 The network architecture shown is applied in the intelligent wildlife protection monitoring system. The system consists of a cloud computing center (as the central device) and multiple edge intelligent devices.

[0103] In the cloud computing center is a server cluster with strong hardware computing and storage capabilities, which can use the pre-selected and collected labeled wildlife data (first training samples). The cloud computing center can use the labeled first training samples to train the hypernetwork to obtain a hypernetwork model. Edge smart devices, including smart cameras installed in different wildlife protection areas, are used to collect and process image data to identify and monitor possible wildlife. Smart cameras are devices that have the ability to run smart models. They have different models, so their computing and storage capabilities vary, but are very limited compared to the cloud computing center. It may not be possible to directly run the hypernetwork model trained by the cloud computing center to complete the task.

[0104] The amount of data collected by smart cameras is huge, and it is expensive to use manpower to label wild animal targets in images; different smart cameras are affected by the environment in which they are located, and the data collected and processed have different scenarios compared to the first training samples in the cloud computing center.

[0105] Different smart cameras need to send their own computing and storage condition information, as well as a series of collected unlabeled image information (the second training samples) to the cloud computing center. Then the cloud computing center trains the super network based on these unlabeled image information and the first training samples included in it, through the training method provided in this application, to train an intelligent model that meets the computing power, storage capacity and scene conditions of each smart camera. In this way, for different cameras, the camera can be equipped with an intelligent model that meets the conditions (its own computing power, storage capacity and scene conditions), and then process the collected images to perform the intelligent monitoring task of wild animals.

[0106] In the above process, if the training method of the present application is not used, different models may need to be trained for different smart cameras to meet their respective computing power requirements and data processing scene requirements; if a new camera device is added to the system, it is necessary to retrain, resulting in low sustainability and flexibility of the system. However, the training method of the present application only trains a hypernetwork model, and a customized smart model can be obtained by collecting the computing power, storage capacity information and collected unlabeled data (i.e., the second training sample) of different smart cameras. The cost of separate training of different models required for different smart cameras and the cost of manual labeling of different scene data collected by different smart cameras are eliminated.

[0107] The above-mentioned intelligent wildlife protection monitoring system can also be replaced by other tasks. For example, monitoring tasks in dangerous environments: such as personnel / object intrusion warning systems in ultra-high voltage power grid substation areas, identification systems for violations of safety regulations on large-scale production lines, etc.; it can also be a smart transportation system composed of different road monitoring equipment, smart traffic command centers and different road electronic camera equipment in smart city public service systems.

[0108] See also Figure 4 The present application embodiment provides a method for training an intelligent model, the method is applied to Figure 3 The network architecture shown in the figure, the execution subject of the method is a central device in the network architecture, and the method is used to train a first neural network to obtain a first intelligent model corresponding to the first neural network and multiple second intelligent models corresponding to multiple sub-networks in the first neural network, including:

[0109] Step 301: Obtain a first network set and a second network set from a first neural network, the first network set includes a first neural network and m-1 second neural networks, m is an integer greater than 1, the second network set includes m second neural networks, the m-1 second neural networks and the m second neural networks are different sub-networks in the first neural network, and the network scale of each neural network in the first network set is greater than the network scale of each neural network in the second network set.

[0110] Step 302: Based on the first sample set and the second sample set, each neural network in the first network set is trained to obtain a first model set, the first sample set includes multiple first training samples, the first training samples include first training data and a label, the label is used to indicate a target in the first training data, the second sample set includes multiple second training samples, the second training samples include second training data, and the first model set includes a first intelligent model obtained by training the first neural network and m-1 second intelligent models obtained by training m-1 second neural networks.

[0111] Optionally, the central device receives data collected by each edge smart device, and uses the data collected by each edge smart device as a second training sample. For example, the data collected by each edge smart device is a picture.

[0112] Step 303: Based on the first sample set, the second sample set and the first model set, each neural network in the second network set is trained to obtain a second model set, where the second model set includes m second intelligent models obtained by training the m second neural networks.

[0113] In an embodiment of the present application, the first neural network is divided into a first network set and a second network set, the first network set includes the first neural network and m-1 second neural networks, and the second network set includes m second neural networks. The first training samples included in the first sample set are labeled training samples, and the second training samples included in the second sample set are unlabeled training samples. Since the scale of the neural networks in the first network set is larger than the scale of the neural networks in the second network set, even if the second training samples in the second sample set are unlabeled training samples, the first sample set and the second sample set can be used to train each neural network in the first network set, and the intelligent model corresponding to each neural network in the first network set is obtained, and the first model set is obtained. Then, the first model set, the first sample set and the second sample set are used to train each neural network in the second network set with a smaller network scale, so that the intelligent model corresponding to each neural network in the second network set can be trained, so that both labeled training samples and unlabeled training samples are included, and intelligent models with superior performance are also trained.

[0114] Among them, the unlabeled training samples are all collected by different edge intelligent devices, and each edge intelligent device is deployed in a different area, so each edge intelligent device is in a different scene. These unlabeled training samples constitute the second sample set, and the embodiment of the present application trains intelligent models of different scales based on the first sample set and the second sample set, so that each intelligent model can meet different scene conditions. Since intelligent models of different scales have different requirements for the computing power and storage capacity of edge intelligent devices, intelligent models of different scales can be deployed on different edge intelligent devices.

[0115] Among them, since the first model set is trained, for each second training sample in the second sample set, the training data in the second training sample is classified and processed by the first model set including m intelligent models, and m processing results corresponding to each second training sample are obtained. Based on the m processing results corresponding to each second training sample, the label corresponding to each second training sample is obtained. In this way, based on the labels corresponding to each first training sample and each second training sample, each neural network in the second network set can be trained, so that each intelligent model in the second model set has superior performance.

[0116] For the above Figure 4 The method 300 is shown, and each step in the method 300 is described below.

[0117] For the above step 301, in step 301, a first network set and a second network set are obtained from the first neural network, the first network set includes a first neural network and m-1 second neural networks, the second network set includes m second neural networks, the m-1 second neural networks and the m second neural networks are different subnetworks of the first neural network, m is an integer greater than 1, and the network scale of each neural network in the first network set is greater than the network scale of each neural network in the second network set.

[0118] In step 301, 2m neural networks are randomly selected in the hypernetwork, the 2m neural networks include a first neural network and 2m-1 sub-networks, the 2m-1 sub-networks are 2m-1 second neural networks, and the first neural network has the largest scale among the 2m neural networks. The m neural networks with the largest scale are selected from the 2m neural networks, the m neural networks include the first neural network and m-1 second neural networks with the largest scale, the first neural network and the selected m-1 second neural networks form a first network set, and the remaining m second neural networks that have not been selected form a second network set.

[0119] Among them, the number of sub-networks included in the first neural network is greater than or equal to 2m-1, so after executing the above steps 301-303, 2m-1 second neural networks can be randomly selected from the first neural network, and the first neural network and the 2m-1 second neural networks are used to continue to execute the above 301-303 operations until each sub-network in the first neural network is trained to obtain an intelligent model corresponding to each sub-network.

[0120] For the above step 302, see Figure 5 In step 302, step 302 is implemented according to the following operations 3021 to 3024. The operations 3021 to 3024 are respectively:

[0121] Step 3021: Based on the first sample set, train the parameters of the convolutional layer and the parameters of the fully connected layer included in the neural network to be trained, the neural network to be trained is any neural network in the first network set, the parameter values ​​of the parameters of the convolutional layer after training are the first parameter values, and the parameter values ​​of the parameters of the fully connected layer after training are the second parameter values.

[0122] The parameters of the training convolutional layer are essentially the parameters of the feature extractor included in the training convolutional layer, and the parameters of the training fully connected layer are essentially the parameters of each classifier included in the training fully connected layer.

[0123] In step 3021, step 3021 is implemented according to the following operations 11 to 14. The operations 11 to 14 are respectively:

[0124] 11: Input each first training sample in the first sample set into the neural network to be trained, so that the neural network to be trained classifies the targets included in the first training data in each first training sample to obtain a sixth output result corresponding to each first training sample.

[0125] For any first training sample, the sixth output result corresponding to the first training sample is used to indicate the target included in the first training data in the first training sample.

[0126] Among them, the neural network to be trained includes a convolutional layer and a fully connected layer. After the first training sample is input into the neural network to be trained, the convolutional layer in the neural network to be trained extracts features from the first training data in the first training sample, and inputs the features into the fully connected layer in the neural network to be trained. Based on the features, the fully connected layer classifies the targets included in the first training data in the first training sample and outputs the processing results. The output processing results are the sixth output results corresponding to the first training sample.

[0127] Among them, each classifier included in the fully connected layer classifies the target included in the first training data in the first training sample based on the feature, so the sixth output result corresponding to the first training sample output by the neural network to be trained includes the processing result output by each classifier.

[0128] For example, the feature extractor of the convolutional layer extracts features from the first training data in the first training sample, inputs the features to each classifier in the neural network to be trained, and each classifier performs classification processing on the targets included in the first training data in the first training sample based on the features, and outputs the processing results. The sixth output result corresponding to the first training sample includes the output results of each classifier.

[0129] 12: Based on the sixth output result corresponding to each first training sample, obtain a fifth loss function value according to the following first loss function.

[0130] The first loss function is:

[0131]

[0132] In the first loss function, s 1 is the fifth loss function value, (x s ,y s ) is the first sample set, (x, y) is the first training sample, x is the first training data in the first training sample, y is the label in the first training sample, K represents the total number of categories, y represents the label, k is the sixth output result corresponding to the first training sample, when k = y 1 [k＝y]The value is 1, C(F(x)) is the classifier function, and z is the number of classifiers included in the fully connected layer.

[0133] Optionally, the first loss function may be a cross entropy loss function or the like.

[0134] 13: When the fifth loss function value is not the minimum value of the first loss function, adjust the parameter values ​​of the convolutional layer parameters and the parameter values ​​of the fully connected layer parameters of the neural network to be trained based on the fifth loss function value, and return to execute 11.

[0135] Adjusting the parameter values ​​of the parameters of the convolutional layer includes adjusting the parameter values ​​of the parameters of the feature extractor of the neural network to be trained. Adjusting the parameter values ​​of the parameters of the fully connected layer includes adjusting the parameter values ​​of the parameters of each classifier of the neural network to be trained.

[0136] It should be noted that, since the neural network to be trained is a network in the first neural network, the feature extractor of the neural network to be trained may also be a feature extractor in other neural networks, so when adjusting the parameters of the feature extractor in the neural network to be trained, the parameters of the feature extractor in other neural networks are also adjusted. Similarly, the classifier of the neural network to be trained may also be a classifier in other neural networks, so when adjusting the parameters of the classifier in the neural network to be trained, the parameters of the classifier in other neural networks are also adjusted.

[0137] 14: When the fifth loss function value is the minimum value of the first loss function, the operation is terminated. For ease of explanation, the parameter value of the parameter of the convolutional layer of the neural network to be trained is referred to as the first parameter value, and the parameter value of the parameter of the fully connected layer of the neural network to be trained is referred to as the second parameter value.

[0138] Step 3022: fix the parameter values ​​of the parameters of the convolutional layer of the neural network to be trained to the first parameter values, and train the parameters of the fully connected layer included in the neural network to be trained based on the first sample set and the second sample set, and the parameter values ​​of the parameters of the fully connected layer after training are changed from the second parameter values ​​to the third parameter values.

[0139] In step 3022, fixing the parameter value of the parameter of the convolutional layer of the neural network to be trained to the first parameter value includes fixing the parameter value of the parameter of the feature extractor of the neural network to be trained to the first parameter value. Based on the first sample set and the second sample set, the parameter of each classifier included in the neural network to be trained is trained, and the parameter value of the parameter of each classifier after training is changed from the second parameter value to the third parameter value.

[0140] The third parameter value of each classifier may or may not be the same.

[0141] In step 3022, the parameters of the fully connected layer included in the neural network to be trained are trained according to the following operations 21 to 26. The operations 21 to 26 are respectively:

[0142] 21: Input each first training sample in the first sample set into the neural network to be trained, so that the neural network to be trained classifies the targets included in the first training data in each first training sample to obtain a first output result corresponding to each first training sample.

[0143] For example, each first training sample in the first sample set is input into the neural network to be trained, so that the neural network to be trained classifies the target included in the first training data in each first training sample to obtain a first output result corresponding to each first training sample.

[0144] For any first training sample, the first output result corresponding to the first training sample is used to indicate the target included in the first training data in the first training sample.

[0145] Among them, after the first training sample is input into the neural network to be trained, the convolution layer (feature extractor) in the neural network to be trained extracts features from the first training data in the first training sample, and inputs the features into the fully connected layer (each classifier in the fully connected layer) in the neural network to be trained. The fully connected layer (each classifier) ​​classifies the targets included in the first training data in the first training sample based on the features, and outputs the processing results. The first output result corresponding to the first training sample includes the output result of each classifier.

[0146] 22: Input each second training sample in the second sample set into the neural network to be trained, so that the neural network to be trained processes the target included in the second training data in each second training sample to obtain a first output result corresponding to each second training sample.

[0147] For any second training sample, the first output result corresponding to the second training sample is used to indicate the target included in the second training data in the second training sample.

[0148] Among them, after the second training sample is input into the neural network to be trained, the convolution layer (feature extractor) in the neural network to be trained extracts features from the second training data in the second training sample, and inputs the features into the fully connected layer (the fully connected layer includes each classifier) ​​in the neural network to be trained. The fully connected layer (each classifier) ​​classifies the targets included in the second training data in the second training sample based on the features, and outputs the processing results. The first output result corresponding to the second training sample includes the output result of each classifier.

[0149] 23: Based on the first output result corresponding to each first training sample and the label included in each first training sample, obtain a first loss function value according to the first loss function.

[0150] In operation 23, s is still used 1 Represents the first loss function value.

[0151] 24: Based on the first output result corresponding to each second training sample, obtain a second loss function value through the following distance loss function, where the first loss function is different from the distance loss function.

[0152] The distance loss function is:

[0153]

[0154] In the distance loss function, s 2 is the second loss function value, {x t} is the second sample set, and (x) is the second training sample.

[0155] 25: When the loss function value of the fully connected layer is not the minimum value, adjust the parameter values ​​of the parameters of the fully connected layer (the multiple classifiers included in the fully connected layer) of the neural network to be trained based on the loss function value of the fully connected layer, and return to execute 3031. The loss function value of the fully connected layer includes the first loss function value and the second loss function value.

[0156] The loss function value of the fully connected layer is equal to s 1 -s 2 ,in,

[0157]

[0158] 26: When the loss function value of the fully connected layer is the minimum, the operation is terminated. For the sake of convenience, the parameter value of the parameter of the convolutional layer (feature extractor) of the neural network to be trained is called the first parameter value, and the parameter value of the parameter of the fully connected layer (multiple classifiers in the fully connected layer) of the neural network to be trained is called the third parameter value.

[0159] Step 3023: fix the parameter values ​​of the parameters of the fully connected layer of the neural network to be trained (multiple classifiers in the fully connected layer) to the third parameter values, and train the parameters of the convolutional layer (feature extractor) of the neural network to be trained based on the first sample set and the second sample set to obtain the intelligent model corresponding to the neural network to be trained.

[0160] In step 3023, the parameters of the convolutional layer (feature extractor) included in the neural network to be trained are trained according to the following operations 31 to 36. The operations 31 to 36 are respectively:

[0161] 31: Input each first training sample in the first sample set into the neural network to be trained, so that the neural network to be trained processes the target included in the first training data in each first training sample to obtain a second output result corresponding to each first training sample.

[0162] For any first training sample, the second output result corresponding to the first training sample is used to indicate the target included in the first training data in the first training sample.

[0163] Among them, after the first training sample is input into the neural network to be trained, the convolution layer (feature extractor) in the neural network to be trained extracts features from the first training data in the first training sample, and inputs the features into the fully connected layer (each classifier in the fully connected layer) in the neural network to be trained. The fully connected layer (each classifier in the fully connected layer) recognizes the target included in the first training data in the first training sample based on the features, and outputs the recognition result. The second output result corresponding to the first training sample includes the output result of the fully connected layer (each classifier in the fully connected layer).

[0164] 32: Input each second training sample in the second sample set into the neural network to be trained, so that the neural network to be trained processes the target included in the second training data in each second training sample to obtain a second output result corresponding to each second training sample.

[0165] For any second training sample, the second output result corresponding to the second training sample is used to indicate the target included in the second training data in the second training sample.

[0166] Among them, after the second training sample is input into the neural network to be trained, the convolution layer (feature extractor) in the neural network to be trained extracts features from the second training data in the second training sample, and inputs the features into the fully connected layer (each classifier in the fully connected layer) in the neural network to be trained. The fully connected layer (each classifier in the fully connected layer) recognizes the target included in the second training data in the second training sample based on the features, and outputs the recognition result. The second output result corresponding to the second training sample includes the output result of each classifier.

[0167] 33: Based on the second output result corresponding to each first training sample and the label included in each first training sample, obtain a third loss function value according to the first loss function.

[0168] In operation 33, s is still used 1 Represents the value of the third loss function.

[0169] 34: Based on the second output result corresponding to each second training sample, obtain a fourth loss function value through the above distance loss function.

[0170] In operation 34, s is still used 2 Represents the value of the fourth loss function.

[0171] 35: When the convolution layer loss function value is not the minimum value, adjust the parameter values ​​of the parameters of the convolution layer (feature extractor) of the neural network to be trained based on the convolution layer loss function value, and return to execute 3041. The convolution layer loss function value includes the third loss function value and the fourth loss function value.

[0172] The convolutional layer loss function value is equal to s 1 +s 2 ,in,

[0173]

[0174] 36: When the convolutional layer loss function value is the minimum, the operation ends and the neural network to be trained at this time is used as the intelligent model.

[0175] For each neural network in the first network set (the first neural network and the m-1 second neural networks), the above steps 3021-3023 are performed on each neural network to obtain m intelligent models. The m intelligent models include the first intelligent model obtained by training the first neural network, and the m-1 second intelligent models obtained by respectively training the m-1 second neural networks.

[0176] For the above step 303, see Figure 6 In step 303, step 303 is implemented according to the following operations 3031 to 3033. The operations 3031 to 3033 are respectively:

[0177] Step 3031: Input each second training sample in the second sample set into each intelligent model in the first model set, and obtain multiple third output results corresponding to the second training sample output by each intelligent model.

[0178] The first model set includes m intelligent models. For each second training sample in the second sample set, the second training sample is input into each of the m intelligent models. Each intelligent model processes the target in the second training data included in the second training sample and outputs the processing result. The processing results output by the m intelligent models are obtained, and a total of m processing results are obtained. The m processing results are used as the m third output results corresponding to the second training sample. For each second training sample in the second sample set, the above operation is performed to obtain the m third output results corresponding to each second training sample in the second sample set.

[0179] Step 3032: Based on the multiple third output results corresponding to each second training sample, obtain the label corresponding to each second training sample respectively.

[0180] For any second training sample, the m third output results corresponding to the second training sample are averaged, or the maximum value or the minimum value is selected from the m third output results. The average value, the maximum value or the minimum value is used as the label corresponding to the second training sample. For each second training sample of the second sample set, the label corresponding to each second training sample is obtained according to the above operation.

[0181] Step 3033: Based on the first sample set, each second training sample and the label corresponding to each second training sample, train each neural network in the second network set to obtain a second model set.

[0182] For any second neural network in the second network set, the second neural network can be trained according to the following operations 41-44. The operations 41-44 are respectively:

[0183] 41: Input each training sample into the second neural network, so that the second neural network processes the target included in the training data in each training sample to obtain a seventh output result corresponding to each training sample, wherein each training sample includes each first training sample in the first sample set and each second training sample in the second sample set.

[0184] For each training sample, the seventh output result corresponding to the training sample is used to indicate the target included in the training data in the training sample.

[0185] Among them, after the training sample is input into the second neural network, the convolution layer (feature extractor) in the second neural network extracts features from the training data in the second training sample, and inputs the features to the fully connected layer (each classifier in the fully connected layer) in the second neural network. The fully connected layer (each classifier in the fully connected layer) processes the targets included in the training data in the training sample based on the features, and outputs the processing results. The seventh output result corresponding to the training sample includes the output result of the fully connected layer.

[0186] 42: Based on the seventh output result corresponding to each training sample and the label corresponding to each training sample, obtain a loss function value according to the first loss function.

[0187] 43: When the loss function value is not the minimum value of the first loss function, adjust the parameter values ​​of the parameters of the second neural network based on the loss function value, and return to execute 41.

[0188] 44: When the loss function value is the minimum value of the first loss function, the operation is terminated and the second neural network at this time is used as the second intelligent model.

[0189] Each second neural network in the second network set can be trained according to the above operations 41-44 to obtain a second intelligent model corresponding to each second neural network, and a total of m second intelligent models can be obtained.

[0190] Through the above steps 301-303, the first intelligent model corresponding to the first neural network and the second intelligent model corresponding to each sub-network in the first neural network are trained. The scale of each second intelligent model is different, and the second intelligent models of different scales are suitable for devices with different hardware performance. Assume that there are n devices with different hardware performance, n is an integer greater than 1, and the number of second intelligent models is large, so a second intelligent model can be selected for each device from the numerous second intelligent models. Figure 7 , the embodiment of the present application provides the following method for selecting an intelligent model, including:

[0191] Step 601: Based on resource information of n devices, n third model sets are determined, and the n devices correspond to the n third model sets one by one.

[0192] Among them, the n devices include a first device, the third model set corresponding to the first device includes at least one second intelligent model, and the resources of the first device meet the resources required by each second intelligent model in the at least one second intelligent model.

[0193] Optionally, the resource information of the first device includes the idle memory size and / or the number of CPU processing cores of the first device, and the resource information of the first device is used to indicate the resources on the device.

[0194] In step 601, based on the resource information of the first device and the scale of each trained second intelligent model, at least one second intelligent model that can be run on the first device is determined, the resources of the first device meet the resources required by each determined second intelligent model, and the model set including the at least one second intelligent model is determined as the third model set corresponding to the first device. In the same way as above, the above operations are performed on the other n-1 second devices respectively to obtain the third model set corresponding to each of the n-1 devices.

[0195] Step 602: Based on the first intelligent model, select a second intelligent model for the first device from the third intelligent model corresponding to the first device.

[0196] In step 602, each second training sample in the second sample set is input into the first intelligent model, and a fourth output result corresponding to each second training sample output by the first intelligent model is obtained. Each second training sample in the second sample set is input into the target intelligent model, and a fifth output result corresponding to each second training sample output by the target intelligent model is obtained. The target intelligent model is a second intelligent model in the third model set corresponding to the first device. Based on the fourth output result and the fifth output result of each second training sample, the difference information between the first intelligent model and the target intelligent model is obtained. In the same manner as described above, the difference information between the first intelligent model and each second intelligent model in the third model set corresponding to the first device is obtained. Based on the difference information between the first intelligent model and each second intelligent model, a second intelligent model is selected from each second intelligent model.

[0197] Optionally, the difference between the fourth output result and the fifth output result of each second training sample is calculated, that is, the difference corresponding to each second training sample is calculated, and based on the difference corresponding to each second training sample, the average difference is calculated, and the average difference is used as the difference information between the first intelligent model and the target intelligent model.

[0198] See also Figure 8 , the embodiment of the present application provides a device 700 for training an intelligent model, the device 700 comprising:

[0199] An acquisition module 701 is used to acquire a first network set and a second network set from a first neural network, wherein the first network set includes a first neural network and m-1 second neural networks, where m is an integer greater than 1, and the second network set includes m second neural networks, wherein the m-1 second neural networks and the m second neural networks are different subnetworks in the first neural network, and the network scale of each neural network in the first network set is greater than the network scale of each neural network in the second network set;

[0200] A first training module 702 is used to train each neural network in the first network set based on a first sample set and a second sample set to obtain a first model set, wherein the first sample set includes a plurality of first training samples, the first training samples include first training data and a label, the label is used to indicate a target in the first training data, the second sample set includes a plurality of second training samples, the second training samples include second training data, and the first model set includes a first intelligent model obtained by training the first neural network and m-1 second intelligent models obtained by training the m-1 second neural networks;

[0201] The second training module 703 is used to train each neural network in the second network set based on the first sample set, the second sample set and the first model set to obtain a second model set, wherein the second model set includes m second intelligent models obtained by training the m second neural networks.

[0202] Optionally, the first training module 702 is used to:

[0203] Based on the first sample set, training parameters of a convolutional layer and parameters of a fully connected layer included in a neural network to be trained, wherein the neural network to be trained is any neural network in the first network set, and parameter values ​​of the parameters of the convolutional layer after training are first parameter values, and parameter values ​​of the parameters of the fully connected layer after training are second parameter values;

[0204] Fixing the parameter value of the parameter of the convolutional layer to the first parameter value, training the parameter of the fully connected layer included in the neural network to be trained based on the first sample set and the second sample set, wherein the parameter value of the parameter of the fully connected layer after training changes from the second parameter value to a third parameter value;

[0205] The parameter values ​​of the parameters of the fully connected layer are fixed to the third parameter values, and the parameters of the convolutional layer of the neural network to be trained are trained based on the first sample set and the second sample set to obtain an intelligent model corresponding to the neural network to be trained.

[0206] Optionally, the first training module 702 is used to:

[0207] Inputting each first training sample in the first sample set into the neural network to be trained, and obtaining a first output result corresponding to each first training sample output by the neural network to be trained, and inputting each second training sample in the second sample set into the neural network to be trained, and obtaining a first output result corresponding to each second training sample;

[0208] Based on the first output result corresponding to each of the first training samples and the label included in each of the first training samples, a first loss function value is obtained through a first loss function, and based on the first output result corresponding to each of the second training samples, a second loss function value is obtained through a distance loss function, where the first loss function is a loss function other than the distance loss function;

[0209] When the fully connected layer loss function value is not a minimum value, the parameter values ​​of the parameters of the fully connected layer are adjusted based on the fully connected layer loss function value, where the fully connected layer loss function value includes the first loss function value and the second loss function value.

[0210] Optionally, the first training module 702 is used to:

[0211] Inputting each first training sample in the first sample set into the neural network to be trained, and obtaining a second output result corresponding to each first training sample output by the neural network to be trained, and inputting each second training sample in the second sample set into the neural network to be trained, and obtaining a second output result corresponding to each second training sample;

[0212] Based on the second output result corresponding to each of the first training samples and the label included in the first training sample, a third loss function value is obtained through the first loss function, and based on the second output result corresponding to each of the second training samples, a fourth loss function value is obtained through the distance loss function;

[0213] When the convolution layer loss function value is not a minimum value, the parameter values ​​of the parameters of the convolution layer are adjusted based on the convolution layer loss function value, and the convolution layer loss function value includes the third loss function value and the fourth loss function value.

[0214] Optionally, the second training module 703 is used to:

[0215] Input each second training sample in the second sample set to each intelligent model in the first model set, and obtain a plurality of third output results corresponding to the second training sample output by each intelligent model;

[0216] Based on the multiple third output results corresponding to each second training sample, respectively obtain a label corresponding to each second training sample;

[0217] Based on the first sample set, each second training sample and a label corresponding to each second training sample, each neural network in the second network set is trained to obtain a second model set.

[0218] Optionally, the device 700 further includes:

[0219] A determination module, configured to determine n third model sets based on resource information of n devices, wherein the n devices correspond one-to-one to the n third model sets, the n devices include a first device, the third model set corresponding to the first device includes at least one second intelligent model, and the resources of the first device meet the resources required by each second intelligent model in the at least one second intelligent model, and n is an integer greater than 1;

[0220] A selection module is used to select a second intelligent model for the first device from a third intelligent model corresponding to the first device based on the first intelligent model.

[0221] Optionally, the selection module is used to:

[0222] Based on the second sample set, obtaining difference information between the first intelligent model and each second intelligent model in a third model set corresponding to the first device;

[0223] Based on the difference information between the first intelligent model and each of the second intelligent models, a second intelligent model is selected from each of the second intelligent models.

[0224] Optionally, the selection module is used to:

[0225] Input each second training sample in the second sample set into the first intelligent model, and obtain a fourth output result corresponding to each second training sample output by the first intelligent model;

[0226] Input each second training sample in the second sample set into a target intelligent model, and obtain a fifth output result corresponding to each second training sample output by the target intelligent model, where the target intelligent model is a second intelligent model in the third model set corresponding to the first device;

[0227] Based on the fourth output result and the fifth output result of each second training sample, the difference information between the first intelligent model and the target intelligent model is obtained.

[0228] In the embodiment of the present application, since the scale of the neural network in the first network set is larger than the scale of the neural network in the second network set, the first sample set and the second sample set can be used to train each neural network in the first network set, and the intelligent model corresponding to each neural network in the first network set can be obtained, thereby obtaining the first model set. Then, the first model set, the first sample set, and the second sample set are used to train each neural network in the second network set with a smaller network scale, so that the intelligent model corresponding to each neural network in the second network set can be trained, so that both labeled training samples and unlabeled training samples are included, and an intelligent model with superior performance is also trained.

[0229] Fig. 9The block diagram of the structure of an electronic device 800 provided by an exemplary embodiment of the present application is shown. The electronic device 800 may be a portable mobile terminal, such as a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 player (Moving Picture Experts Group Audio Layer IV), a laptop computer or a desktop computer. The electronic device 800 may also be called a user device, a portable terminal, a laptop terminal, a desktop terminal or other names.

[0230] Typically, the electronic device 800 includes a processor 801 and a memory 802 .

[0231] The processor 801 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 801 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 801 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 801 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 801 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0232] The memory 802 may include one or more computer-readable storage media, which may be non-transitory. The memory 802 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 802 is used to store at least one instruction, which is used to be executed by the processor 801 to implement the method for training an intelligent model provided in the method embodiment of the present application.

[0233] In some embodiments, the electronic device 800 may further optionally include: a peripheral device interface 803 and at least one peripheral device. The processor 801, the memory 802 and the peripheral device interface 803 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 803 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 804, a display screen 805, a camera assembly 806, an audio circuit 807, a positioning assembly 808 and a power supply 809.

[0234] The peripheral device interface 803 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 801 and the memory 802. In some embodiments, the processor 801, the memory 802, and the peripheral device interface 803 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 801, the memory 802, and the peripheral device interface 803 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0235] The radio frequency circuit 804 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 804 communicates with communication networks and other communication devices through electromagnetic signals. The radio frequency circuit 804 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the radio frequency circuit 804 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The radio frequency circuit 804 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: the World Wide Web, a metropolitan area network, an intranet, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 804 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0236] The display screen 805 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos and any combination thereof. When the display screen 805 is a touch display screen, the display screen 805 also has the ability to collect touch signals on the surface or above the surface of the display screen 805. The touch signal can be input to the processor 801 as a control signal for processing. At this time, the display screen 805 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 805 can be one, set on the front panel of the electronic device 800; in other embodiments, the display screen 805 can be at least two, respectively set on different surfaces of the electronic device 800 or in a folding design; in other embodiments, the display screen 805 can be a flexible display screen, set on the curved surface or folding surface of the electronic device 800. Even, the display screen 805 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 805 can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode, organic light-emitting diode).

[0237] The camera assembly 806 is used to capture images or videos. Optionally, the camera assembly 806 includes a front camera and a rear camera. Typically, the front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 806 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0238] The audio circuit 807 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 801 for processing, or input them into the radio frequency circuit 804 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the electronic device 800. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 801 or the radio frequency circuit 804 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 807 may also include a headphone jack.

[0239] Positioning component 808 is used to locate the current geographic location of electronic device 800 to implement navigation or LBS (Location Based Service). Positioning component 808 can be a positioning component based on the US GPS (Global Positioning System), China's Beidou system or Russia's Galileo system.

[0240] The power supply 809 is used to power various components in the electronic device 800. The power supply 809 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 809 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery charged through a wired line, and a wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0241] In some embodiments, the electronic device 800 further includes one or more sensors 810 , including but not limited to: an acceleration sensor 811 , a gyroscope sensor 812 , a pressure sensor 813 , a fingerprint sensor 814 , an optical sensor 815 , and a proximity sensor 816 .

[0242] The acceleration sensor 811 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the electronic device 800. For example, the acceleration sensor 811 can be used to detect the components of gravity acceleration on the three coordinate axes. The processor 801 can control the display screen 805 to display the user interface in a horizontal view or a vertical view according to the gravity acceleration signal collected by the acceleration sensor 811. The acceleration sensor 811 can also be used for collecting game or user motion data.

[0243] The gyro sensor 812 can detect the body direction and rotation angle of the electronic device 800, and the gyro sensor 812 can cooperate with the acceleration sensor 811 to collect the user's 3D actions on the electronic device 800. The processor 801 can implement the following functions based on the data collected by the gyro sensor 812: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0244] The pressure sensor 813 can be set on the side frame of the electronic device 800 and / or the lower layer of the display screen 805. When the pressure sensor 813 is set on the side frame of the electronic device 800, it can detect the user's holding signal of the electronic device 800, and the processor 801 performs left and right hand recognition or shortcut operations according to the holding signal collected by the pressure sensor 813. When the pressure sensor 813 is set on the lower layer of the display screen 805, the processor 801 controls the operability controls on the UI interface according to the user's pressure operation on the display screen 805. The operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0245] The fingerprint sensor 814 is used to collect the user's fingerprint, and the processor 801 identifies the user's identity based on the fingerprint collected by the fingerprint sensor 814, or the fingerprint sensor 814 identifies the user's identity based on the collected fingerprint. When the user's identity is identified as a trusted identity, the processor 801 authorizes the user to perform relevant sensitive operations, which include unlocking the screen, viewing encrypted information, downloading software, paying, and changing settings. The fingerprint sensor 814 can be set on the front, back, or side of the electronic device 800. When a physical button or a manufacturer logo is set on the electronic device 800, the fingerprint sensor 814 can be integrated with the physical button or the manufacturer logo.

[0246] The optical sensor 815 is used to collect the ambient light intensity. In one embodiment, the processor 801 can control the display brightness of the display screen 805 according to the ambient light intensity collected by the optical sensor 815. Specifically, when the ambient light intensity is high, the display brightness of the display screen 805 is increased; when the ambient light intensity is low, the display brightness of the display screen 805 is reduced. In another embodiment, the processor 801 can also dynamically adjust the shooting parameters of the camera assembly 806 according to the ambient light intensity collected by the optical sensor 815.

[0247] The proximity sensor 816, also called a distance sensor, is usually disposed on the front panel of the electronic device 800. The proximity sensor 816 is used to collect the distance between the user and the front of the electronic device 800. In one embodiment, when the proximity sensor 816 detects that the distance between the user and the front of the electronic device 800 is gradually decreasing, the processor 801 controls the display screen 805 to switch from the screen-on state to the screen-off state; when the proximity sensor 816 detects that the distance between the user and the front of the electronic device 800 is gradually increasing, the processor 801 controls the display screen 805 to switch from the screen-off state to the screen-on state.

[0248] Those skilled in the art will understand that Fig. 9 The structure shown in the figure does not constitute a limitation on the electronic device 800, and may include more or less components than those shown in the figure, or combine some components, or adopt a different component arrangement.

[0249] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the application disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0250] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for training an intelligent model, characterized in that: The method is applied to a central device of a network architecture, the network architecture further comprising a plurality of edge intelligent devices, the central device comprising a first sample set, the first sample set comprising a plurality of first training samples, the first training samples comprising first training data and a label, the label being used to indicate a target in the first training data, the first training data being a picture, the method comprising: The central device obtains a second sample set, where the second sample set includes a plurality of second training samples, where the second training samples include second training data, and where the second training samples are pictures collected by the edge smart device; The central device acquires a first network set and a second network set from a first neural network, the first network set includes a first neural network and m-1 second neural networks, m is an integer greater than 1, the second network set includes m second neural networks, the m-1 second neural networks and the m second neural networks are different subnetworks in the first neural network, and the network scale of each neural network in the first network set is greater than the network scale of each neural network in the second network set; The central device trains each neural network in the first network set based on the first sample set and the second sample set to obtain a first model set, wherein the first model set includes a first intelligent model obtained by training the first neural network and m-1 second intelligent models obtained by training the m-1 second neural networks; The central device trains each neural network in the second network set based on the first sample set, the second sample set and the first model set to obtain a second model set, wherein the second model set includes m second intelligent models obtained by training the m second neural networks, and each second intelligent model has a different scale; The central device determines the second intelligent model corresponding to each edge intelligent device based on the resource information of each edge intelligent device, the resources of each edge intelligent device respectively satisfy the second intelligent model corresponding to each edge intelligent device, and installs the second intelligent model corresponding to each edge intelligent device on each edge intelligent device respectively.

2. The method according to claim 1, characterized in that The step of training each neural network in the first network set based on the first sample set and the second sample set to obtain a first model set includes: Based on the first sample set, training parameters of a convolutional layer and parameters of a fully connected layer included in a neural network to be trained, wherein the neural network to be trained is any neural network in the first network set, and parameter values ​​of the parameters of the convolutional layer after training are first parameter values, and parameter values ​​of the parameters of the fully connected layer after training are second parameter values; Fixing the parameter value of the parameter of the convolutional layer to the first parameter value, training the parameter of the fully connected layer included in the neural network to be trained based on the first sample set and the second sample set, wherein the parameter value of the parameter of the fully connected layer after training changes from the second parameter value to a third parameter value; The parameter values ​​of the parameters of the fully connected layer are fixed to the third parameter values, and the parameters of the convolutional layer of the neural network to be trained are trained based on the first sample set and the second sample set to obtain an intelligent model corresponding to the neural network to be trained.

3. The method according to claim 2, characterized in that The step of training the parameters of the fully connected layer of the neural network to be trained based on the first sample set and the second sample set includes: Inputting each first training sample in the first sample set into the neural network to be trained, and obtaining a first output result corresponding to each first training sample output by the neural network to be trained, and inputting each second training sample in the second sample set into the neural network to be trained, and obtaining a first output result corresponding to each second training sample; Based on the first output result corresponding to each of the first training samples and the label included in each of the first training samples, a first loss function value is obtained through a first loss function, and based on the first output result corresponding to each of the second training samples, a second loss function value is obtained through a distance loss function, where the first loss function is a loss function other than the distance loss function; When the fully connected layer loss function value is not a minimum value, the parameter values ​​of the parameters of the fully connected layer are adjusted based on the fully connected layer loss function value, where the fully connected layer loss function value includes the first loss function value and the second loss function value.

4. The method according to claim 3, characterized in that The step of training the parameters of the convolutional layer of the neural network to be trained based on the first sample set and the second sample set includes: Inputting each first training sample in the first sample set into the neural network to be trained, and obtaining a second output result corresponding to each first training sample output by the neural network to be trained, and inputting each second training sample in the second sample set into the neural network to be trained, and obtaining a second output result corresponding to each second training sample; Based on the second output result corresponding to each of the first training samples and the label included in the first training sample, a third loss function value is obtained through the first loss function, and based on the second output result corresponding to each of the second training samples, a fourth loss function value is obtained through the distance loss function; When the convolution layer loss function value is not a minimum value, the parameter values ​​of the parameters of the convolution layer are adjusted based on the convolution layer loss function value, and the convolution layer loss function value includes the third loss function value and the fourth loss function value.

5. The method according to claim 1, characterized in that The step of training each neural network in the second network set based on the first sample set, the second sample set and the first model set to obtain a second model set includes: Input each second training sample in the second sample set to each intelligent model in the first model set, and obtain multiple third output results corresponding to the second training sample output by each intelligent model; Based on the multiple third output results corresponding to each second training sample, respectively obtain a label corresponding to each second training sample; Based on the first sample set, each second training sample and a label corresponding to each second training sample, each neural network in the second network set is trained to obtain a second model set.

6. The method according to claim 1, characterized in that The determining, based on the resource information of each edge intelligent device, a second intelligent model corresponding to each edge intelligent device includes: Based on resource information of n devices, determine n third model sets, the n devices correspond one-to-one to the n third model sets, the n devices include a first device, the third model set corresponding to the first device includes at least one second intelligent model, resources of the first device meet resources required by each second intelligent model in the at least one second intelligent model, and n is an integer greater than 1; Based on the first intelligent model, a second intelligent model is selected for the first device from a third model set corresponding to the first device.

7. The method according to claim 6, characterized in that The selecting, based on the first intelligent model, a second intelligent model for the first device from a third model set corresponding to the first device includes: Based on the second sample set, obtaining difference information between the first intelligent model and each second intelligent model in a third model set corresponding to the first device; Based on the difference information between the first intelligent model and each of the second intelligent models, a second intelligent model is selected from each of the second intelligent models.

8. The method according to claim 7, characterized in that The acquiring, based on the second sample set, difference information between the first intelligent model and each second intelligent model in a third model set corresponding to the first device includes: Input each second training sample in the second sample set into the first intelligent model, and obtain a fourth output result corresponding to each second training sample output by the first intelligent model; Input each second training sample in the second sample set into a target intelligent model, and obtain a fifth output result corresponding to each second training sample output by the target intelligent model, where the target intelligent model is a second intelligent model in the third model set corresponding to the first device; Based on the fourth output result and the fifth output result of each second training sample, the difference information between the first intelligent model and the target intelligent model is obtained.

9. The method according to any one of claims 1 to 8, characterized in that The network architecture is a system architecture of central computing-edge devices, each of the edge intelligent devices is deployed in a different area, and the computing power and storage capacity of the central device are respectively greater than the computing power and storage capacity of each of the edge intelligent devices.

10. A device for training an intelligent model, characterized in that: The device is located in a network architecture, the network architecture further includes a plurality of edge intelligent devices, the device includes a first sample set, the first sample set includes a plurality of first training samples, the first training samples include first training data and a label, the label is used to indicate a target in the first training data, the first training data is a picture, and the device includes: The apparatus is used to obtain a second sample set, where the second sample set includes a plurality of second training samples, where the second training samples include second training data, and where the second training samples are pictures collected by the edge smart device; An acquisition module is used to acquire a first network set and a second network set from a first neural network, wherein the first network set includes a first neural network and m-1 second neural networks, where m is an integer greater than 1, and the second network set includes m second neural networks, wherein the m-1 second neural networks and the m second neural networks are different subnetworks in the first neural network, and the network scale of each neural network in the first network set is greater than the network scale of each neural network in the second network set; A first training module is used to train each neural network in the first network set based on the first sample set and the second sample set to obtain a first model set, wherein the first model set includes a first intelligent model obtained by training the first neural network and m-1 second intelligent models obtained by training the m-1 second neural networks; A second training module is used to train each neural network in the second network set based on the first sample set, the second sample set and the first model set to obtain a second model set, wherein the second model set includes m second intelligent models obtained by training the m second neural networks, and each second intelligent model has a different scale; The device is also used to determine the second intelligent model corresponding to each edge intelligent device based on the resource information of each edge intelligent device, the resources of each edge intelligent device respectively satisfy the second intelligent model corresponding to each edge intelligent device, and install the second intelligent model corresponding to each edge intelligent device on each edge intelligent device respectively.

Images