A method and apparatus for generating an image processing model, a processing method, and a terminal device

Abstract

The present application discloses a method, apparatus, processing method and terminal device for generating an image processing model. The method includes training a first network model based on an initial data set and a migration data set; determining a second network model based on the first network model, training the second network model based on a second data set to obtain a target network model, and using the feature extraction module of the target network model as the image processing model. The present application utilizes the transferability of image visual features, learns a first network model from the initial data set and the migration data set, and trains a second network model that inherits the feature extraction module of the first network model using the migration data set. The feature extraction module of the trained target network model is used as the image processing model. In this way, for different image processing tasks, this image processing model can be directly used as the feature extraction module without training the feature extraction module, reducing the training cost of the image processing model.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and particularly relates to a method for generating an image processing model, an apparatus, a processing method, and a terminal device. Background Art

[0002] With the gradual maturity of deep learning technology, convolutional neural network models are widely used in image processing tasks (such as image classification tasks, image detection tasks, and image segmentation tasks, etc.), and learning image features is required in the process of applying to each image processing task. At present, when applying a convolutional neural network model to an image processing task, the image processing task is regarded as an independent task, and the neural network model applied to the image processing task is retrained, which increases the training cost of the neural network model. Summary of the Invention

[0003] The technical problem to be solved by this application is to provide a method for generating an image processing model, an apparatus, a processing method, and a terminal device in view of the deficiencies of the prior art.

[0004] To solve the above technical problem, in the first aspect of the embodiments of this application, a method for generating an image processing model is provided, and the method includes:

[0005] Obtain an initial data set and a transfer data set, and train a preset network model based on the initial data set and the transfer data set to obtain a first network model, where the preset network model includes a feature extraction module;

[0006] Determine a second network model based on the feature extraction module, where the second network model includes the feature extraction module;

[0007] Train the second network model based on the transfer data set to obtain a target network model, and use the feature extraction module of the target network model as an image processing model.

[0008] In the method for generating an image processing model, the step of obtaining an initial data set and a transfer data set, and training a preset network model based on the initial data set and the transfer data set to obtain a first network model specifically includes:

[0009] Obtain an initial data set and a transfer data set;

[0010] Determine a training data set according to the initial data set and the transfer data set;

[0011] Train a preset network model based on the training data set to obtain a first network model.

[0012] The method for generating the image processing model, wherein determining the training dataset according to the initial dataset and the migration dataset specifically includes:

[0013] Obtain a number of preset angles;

[0014] For each preset angle, rotate the initial images in the initial dataset and the migration images in the migration dataset according to this preset angle respectively to obtain the target initial dataset corresponding to the initial dataset and the target migration dataset corresponding to the migration dataset;

[0015] Generate the training dataset according to the initial dataset, each target initial dataset, the migration dataset and each target migration dataset.

[0016] The method for generating the image processing model, wherein the model parameters of the feature extraction module in the second network model are the same as the model parameters in the first network model.

[0017] The method for generating the image processing model, wherein the first network model includes a first fully connected module, and determining the second network model based on the feature extraction module specifically includes:

[0018] Obtain the feature extraction module in the first network model and determine a second fully connected module, wherein the output dimension of the second fully connected module is less than the output dimension of the first fully connected module;

[0019] Cascade the feature extraction module and the second fully connected module to obtain the second network model.

[0020] The method for generating the image processing model, wherein before training the second network model with the migration dataset to obtain the target network model, the method further includes:

[0021] Divide the migration images in the migration dataset into several sub-datasets according to their respective corresponding class labels, wherein the class labels of the migration images in the several sub-datasets are the same;

[0022] For every two sub-datasets in the several sub-datasets, generate a candidate dataset according to the first sub-dataset and the second sub-dataset in the two sub-datasets;

[0023] Use all the generated candidate datasets as the migration dataset.

[0024] The method for generating the image processing model, wherein generating the candidate dataset according to the first sub-dataset and the second sub-dataset in the two sub-datasets specifically includes:

[0025] For each migrated image in the first sub-dataset and the second sub-dataset, obtain the edge pixel points of the migrated image, and select several target edge pixel points from the obtained edge pixel points; select the first image corresponding to each target edge pixel point in the migrated image, where each target edge pixel point is a pixel point in its corresponding first image; determine the second image corresponding to each first image according to the migrated image, where the second image is the image area of the migrated image excluding the first image.

[0026] Generate a candidate dataset based on all the first images and all the second images selected from the migrated images.

[0027] The method for generating the image processing model, wherein training the second network model based on the migration dataset to obtain the target network model includes:

[0028] Select a candidate dataset from the migration dataset, and train the second network model based on the candidate dataset to obtain a candidate network model;

[0029] Obtain the network parameters of the feature extraction module in the candidate network model, and configure the network parameters in the second network model to update the second network model;

[0030] Continue the step of selecting a candidate dataset from the migration dataset until all the candidate datasets in the migration dataset are selected, where the candidate datasets selected each time are different from each other.

[0031] The method for generating the image processing model, wherein after training the second network model based on the migration dataset to obtain the target network model and using the feature extraction module of the target network model as the image processing model, the method further includes:

[0032] Obtain the target migration dataset, and determine the third fully connected module corresponding to the target migration dataset;

[0033] Obtain the image processing model, and use the image processing model as the target feature extraction module corresponding to the target migration dataset;

[0034] Cascade the target feature extraction module and the third fully connected module to obtain the third network model corresponding to the target migration dataset;

[0035] Train the third network model based on the target migration dataset to obtain the target image processing model corresponding to the target migration dataset.

[0036] Method for generating the image processing model, wherein the network parameters of the target feature extraction module in the target image processing model are the same as those of the target feature extraction module in the third network model.

[0037] In a second aspect of the embodiments of the present application, a processing method is provided. The method is applied to the image processing model as described above, and the method includes:

[0038] Obtain an image to be processed;

[0039] Input the image to be processed into the image processing model, and output an output image corresponding to the image to be processed through the image processing model.

[0040] In a third aspect of the embodiments of the present application, a generating device is provided. The generating device includes:

[0041] An obtaining module, configured to obtain an initial data set and a migration data set, and train a preset network model based on the initial data set and the migration data set to obtain a first network model, wherein the preset network model includes a feature extraction module;

[0042] A determining module, configured to determine a second network model based on the feature extraction module, wherein the second network model includes the feature extraction module;

[0043] A training module, configured to train the second network model based on the migration data set to obtain a target network model, and use the feature extraction module of the target network model as the image processing model.

[0044] In a fourth aspect of the embodiments of the present application, a computer-readable storage medium is provided. The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the method for generating the image processing model as described above, or to implement the steps in the processing method as described above.

[0045] In a fifth aspect of the embodiments of the present application, a terminal device is provided. It includes: a processor, a memory, and a communication bus; a computer-readable program executable by the processor is stored on the memory;

[0046] The communication bus realizes the connection and communication between the processor and the memory;

[0047] When the processor executes the computer-readable program, it implements the steps in the method for generating the image processing model as described above, or implements the steps in the processing method as described above.

[0048] Beneficial effects: Compared with the prior art, the present application provides a method, an apparatus, a processing method and a terminal device for generating an image processing model. The method includes obtaining an initial data set and a migration data set, and training a preset network model based on the initial data set and the migration data set to obtain a first network model; determining a second network model based on the feature extraction module, and generating a second data set based on the migration data set; training the second network model based on the second data set to obtain a target network model, and using the feature extraction module of the target network model as the image processing model. The present application utilizes the transferability of image visual features to learn the first network model from the initial data set and the migration data set, and trains the second network model that inherits the feature extraction module of the first network model using the migration data set, and uses the feature extraction module of the trained target network model as the image processing model, so that the image processing model can learn image features. In this way, for different image processing tasks that need to learn image features, the image processing model can be directly used as the feature extraction module without training the feature extraction module, reducing the training cost of the image processing model. Description of the Drawings

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0050] Figure 1 It is a flowchart of the method for generating an image processing model provided by the present application.

[0051] Figure 2 It is a flow example diagram of the method for generating an image processing model provided by the present application.

[0052] Figure 3 It is a schematic diagram of the principle of the training data set determination process in the method for generating an image processing model provided by the present application.

[0053] Figure 4 It is a schematic diagram of the principle of the candidate data set determination process in the method for generating an image processing model provided by the present application.

[0054] Figure 5 It is a flowchart of the processing method provided by the present application.

[0055] Figure 6 It is a schematic diagram of the structure principle of the generating device provided by the present application.

[0056] Figure 7 It is a schematic diagram of the structure principle of the terminal device provided by the present application. Detailed implementation manners

[0057] The present application provides a method, an apparatus, a processing method and a terminal device for generating an image processing model. To make the objectives, technical solutions and effects of the present application clearer and more definite, the following further elaborates on the present application by way of examples with reference to the accompanying drawings. It should be understood that the specific examples described herein are only used to explain the present application and are not used to limit the present application.

[0058] Those skilled in the art of the present technology can understand that, unless specifically stated, the singular forms "a", "an", "the" and "said" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the specification of the present application means that there are the described features, integers, steps, operations, elements and / or components, but does not exclude the existence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups. It should be understood that when we say an element is "connected" or "coupled" to another element, it can be directly connected or coupled to other elements, or there may also be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The phrase "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0059] Those skilled in the art of the present technology can understand that, unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meaning as the general understanding of those of ordinary skill in the field to which the present application belongs. It should also be understood that terms such as those defined in a general dictionary should be understood to have a meaning consistent with the meaning in the context of the prior art, and will not be interpreted with an idealized or overly formal meaning unless specifically defined as here.

[0060] The inventors have found through research that the technology of deep learning has been gradually matured, and the convolutional neural network model CNN is widely used in image processing tasks (such as image classification tasks, image detection tasks, and image segmentation tasks, etc.). Although it has been proven effective to extract image features through the CNN model to solve image tasks, the training process of the CNN model is highly dependent on training data. For example, when the training data set is insufficient, the model accuracy of the CNN model will be affected. In addition, currently when applying the CNN model to image vision tasks, for different image vision tasks, the CNN model needs to be retrained. Even for the same image vision task, when a new category appears in the image vision task or new data appears, the model also needs to be retrained. This not only increases the training cost of the CNN model but also reduces the training efficiency of the CNN model.

[0061] However, for different image vision tasks, human vision has good transfer learning ability for different vision tasks. For example, while identifying apples in an image (classification), the position of the apples in the image can be determined. In other words, the image features determined in the task of classifying the items carried in the image can be used in the task of detecting the items carried in the image. Thus, based on the transfer learning characteristics of image vision, after obtaining the initial dataset and the transfer dataset in the embodiments of the present application, a first network model is trained based on the initial dataset and the transfer dataset; a second network model is determined based on the first network model, and the second network model is trained based on the second dataset to obtain the target network model, and the feature extraction module of the target network model is used as the image processing model. The present application utilizes the transferability of image vision features, learns the first network model from the initial dataset and the transfer dataset, and trains the second network model inheriting the feature extraction module of the first network model with the transfer dataset, and uses the feature extraction module of the trained target network model as the image processing model. In this way, for different image processing tasks, this image processing model can be directly used as the feature extraction module without training the feature extraction module, reducing the training cost of the image processing model.

[0062] The following further describes the application content by describing the embodiments in conjunction with the accompanying drawings.

[0063] Figure 1 It is a schematic flowchart of a method for generating an image processing model provided in this embodiment. This method can be executed by a system upgrade device, and the device can be implemented by software and applied to electronic terminals such as a PC, a server, or a personal digital assistant. Refer to Figure 1 and Figure 2 , the method for generating an image processing model provided in this embodiment specifically includes:

[0064] S10. Obtain an initial dataset and a transfer dataset, and train a preset network model based on the initial dataset and the transfer dataset to obtain a first network model, where the preset network model includes a feature extraction module.

[0065] Specifically, both the initial data set and the migration data set are used for image vision-based image processing tasks, where the image processing tasks corresponding to the initial data set are different from those corresponding to the migration data set. For example, the image processing task corresponding to the initial data set is a classification task of classifying objects in an image (for example, the image is an image carrying an apple, and the processing result obtained through this image processing task is that the object in the image is an apple), and the image processing task corresponding to the migration data set is a detection task of detecting the position of an object in the image (for example, the image is an image carrying an apple, and the processing result obtained through this image processing task is the position information of the apple in the image).

[0066] In an implementation manner of this embodiment, the initial image set includes several initial image groups, and the corresponding categories of the initial image groups are different. The different corresponding categories of the initial image groups are the categories corresponding to the image processing tasks of the initial image groups. The migration image set includes several migration image groups, and the corresponding categories of the migration image groups are different. The different corresponding categories of the initial image groups are the categories corresponding to the image processing tasks of the initial image groups. For example, if the image processing task corresponding to the initial image group is panel defect morphology detection, then the category is the panel defect morphology category; if the image processing task corresponding to the migration data set is panel defect cause detection, then the category is the panel defect cause category. In a specific implementation manner, the number of categories corresponding to the initial images is different from the number of categories corresponding to the migration data, so that the image processing model obtained by subsequent training can be applicable to image processing tasks with different numbers of categories. Of course, in actual applications, the number of images in the initial image set is greater than the number of images in the migration data set.

[0067] In an implementation manner of this embodiment, each initial image in the initial image set and each migration image in the migration data set can be obtained by being captured by an imaging system (such as a camera, a video camera, etc.), or can also be images captured by other external devices (such as a digital camera, etc.) and stored in a terminal device, or can also be images sent to the terminal device through the cloud. Each initial image includes the category annotation corresponding to the initial image, and each migration image includes the category annotation corresponding to the migration image.

[0068] In an implementation manner of this embodiment, the steps of obtaining the initial data set and the migration data set and training a preset network model based on the initial data set and the migration data set to obtain a first network model specifically include:

[0069] Obtain the initial data set and the migration data set;

[0070] Determine a training data set according to the initial data set and the migration data set;

[0071] Train a preset network model based on the training data set to obtain a first network model.

[0072] Specifically, the preset network model is an untrained network model, the training data set is the training sample data set corresponding to the preset network model, and the first network model is a network model obtained by training the preset network model. Among them, the model structure of the first network model is the same as that of the preset network model, and the model parameters of the first network model are different from those of the preset network model. The model parameters of the preset network model are initial model parameters, and the model parameters of the first network model are the model parameters obtained after training the preset network model with the training data. In this embodiment, by jointly training the first network model with the initial data set and the transfer data set, the first network model can learn the common features of the initial data set and the transfer data set, thereby improving the generality of the first network model.

[0073] The preset network model includes a feature map extraction module, a first fully connected module, and a classification module, and the feature map extraction module, the first fully connected module, and the classification module are cascaded in sequence. Since the model structure of the first network model is the same as that of the preset network model, the first network model includes a feature map extraction module, a first fully connected module, and a classification module. In the first network model, the input item of the feature extraction module is the training image in the training image set, the output item of the feature extraction module is the input item of the first fully connected model, and the output item of the first fully connected module is the category corresponding to the training image. In one implementation manner of this embodiment, the feature extraction module is a convolutional network model, the output item of the first fully connected layer is an 8-dimensional vector, where the feature extraction module is the backbone network of the first network model, the first fully connected layer is the head network of the first network model, and the classification module uses the softmax function.

[0074] The training data set is used to train the preset network model, and the training data set includes each initial image in the initial image set and each transfer image in the transfer image set. In addition, in order to improve the diversity of the training image set, when determining the training data set based on the initial data set and the transfer data set, the initial data set and the transfer data set can be enhanced. Correspondingly, in one implementation manner of this embodiment, determining the training data set according to the initial data set and the transfer data set specifically includes:

[0075] Obtain a number of preset angles;

[0076] For each preset angle, rotate the initial images in the initial dataset and the migration images in the migration dataset respectively according to the preset angle, so as to obtain the target initial dataset corresponding to the initial dataset and the target migration dataset corresponding to the migration dataset;

[0077] Generate the training dataset according to the initial dataset, each target initial dataset, the migration dataset and each target migration dataset.

[0078] Specifically, the several preset angles are all preset in advance for rotating the initial images and the migration images, and each preset angle in the several preset angles is different from each other. For example, the several preset angles include preset angle A and preset angle B, preset angle A is 90°, preset angle B is 270°, etc. After the several preset angles are sorted from small to large, the angle difference between any two adjacent preset angles is the same. For example, the preset angles include preset angle A, preset angle B and preset angle C, and preset angle A, preset angle B and preset angle C are arranged from small to large as: preset angle A, preset angle C, preset angle B; then preset angle C - preset angle A = preset angle B - preset angle C. In a specific implementation manner of this embodiment, as Figure 3 shown, the several preset angles include three preset angles, which are 90°, 180° and 270° respectively. The training dataset obtained according to the three preset angles includes the initial dataset I, the target initial image set I-90 obtained by rotating each initial image in the initial dataset by 90°, the target initial image set I-180 obtained by rotating each initial image in the initial dataset by 180°, the target initial image set I-270 obtained by rotating each initial image in the initial dataset by 270°, the migration image set L, the target migration image set L-90 obtained by rotating each migration image in the migration image set by 90°, the target migration image set L-180 obtained by rotating each migration image in the migration image set by 180, and the target migration image set L-270 obtained by rotating each migration image in the migration image set by 270°.

[0079] Further, for each preset angle, each initial image in the initial dataset is rotated by the preset angle, so that each initial image in the initial image set corresponds to a rotated initial image, and each migration image in the migration dataset is rotated by the preset angle, so that each migration image in the migration image set corresponds to a rotated migration image. Thus, for each preset angle, the initial dataset generates a target initial dataset based on the preset angle, and the migration dataset generates a target migration dataset based on the preset angle. In other words, the number of target initial datasets corresponding to the initial dataset is equal to the number of preset angles, and the number of target migration datasets corresponding to the migration dataset is equal to the number of preset angles. For example, if the number of preset angles is 3, then the number of target initial datasets is 3, and the number of target migration datasets is 3.

[0080] S20. Determine the second network model based on the feature extraction module.

[0081] Specifically, the second network model includes a feature extraction module, and the model structure of this feature extraction module is the same as that of the feature extraction module in the first network model, and the model parameters of the feature extraction module in the second network model are the same as the model parameters in the first network model. It can be understood that the second network model inherits the feature map extraction module and model parameters of the first network model, so that the model parameters of the feature extraction module in the second network model are trained, and the network parameters of other network structures in the second network model except the feature extraction module are untrained network parameters.

[0082] In an implementation manner of this embodiment, the determining the second network model based on the feature extraction module specifically includes:

[0083] Obtain the feature extraction module in the first network model and determine the second fully connected module;

[0084] Cascade the feature extraction module and the second fully connected module to obtain the second network model.

[0085] Specifically, the second fully-connected module is the network head of the second network model, the obtained feature extraction module is the backbone network of the second network model, the input item of the feature extraction module is the input item of the second network model, and the output item of the feature extraction module is the input item of the second fully-connected module. Among them, the network parameters of the feature extraction module are inherited from the network parameters of the feature extraction module in the first network model, and the network parameters of the second fully-connected module are initial parameters. It can be seen that the network parameters of the feature extraction module in the second network model are trained network parameters, and the network parameters of the second fully-connected model are initial parameters. In this way, when the training image is input into the second network model, the feature extraction module in the second network model can transfer and learn the image features learned by the feature extraction model in the first network model, thereby improving the accuracy of the extracted image features and improving the training speed of the second network model. In practical applications, the second network model includes a second classification model, and the second classification module can determine the category corresponding to the training image based on the output item of the second fully-connected module, where the second classification module can be configured with a softmax function.

[0086] In an implementation manner of this embodiment, in order to simplify the number of parameters of the image processing model determined based on the second network model subsequently, the dimension of the output item of the second fully-connected module can be less than the dimension of the output item of the first fully-connected module. Thus, when determining the second fully-connected module, the first dimension of the output item of the first fully-connected module can be obtained, and the second dimension of the output item of the second fully-connected module can be determined based on this first dimension, so that the second dimension is less than the first dimension. For example, if the first dimension is 8, then the second dimension can be a positive integer less than 8. For example, the second dimension is 4, and the feature vector of the second fully-connected module attribute can be (1, 1, 4).

[0087] S30. Train the second network model based on the migration data set to obtain a target network model, and use the feature extraction module of the target network model as the image processing model.

[0088] Specifically, the target network model is a trained network model obtained by training the second network model. The target network model includes a feature extraction module. The model structure of the feature extraction module in the target network model is the same as that of the feature extraction module in the second network model, but the network parameters of the feature extraction module in the target network model are different from those of the feature extraction module in the second network model. Among them, the network parameters of the feature extraction module in the target network model are trained, and the network parameters of the feature extraction module in the second network model are inherited from the feature extraction module in the first network model. In this way, migrating the feature extraction module of the first network model to the second network model can improve the feature generalization ability of the feature extraction module in the second network model, and improve the training convergence efficiency and prediction accuracy of the second network model.

[0089] The image processing model is the feature extraction module in the target network model. The image processing module is used to extract the feature map of the image to be detected, and the network parameters of the image processing model are trained. When subsequently establishing a network model for performing image vision-based image processing tasks, this image processing model can be directly used as the feature extraction module of the network model, and when training the network model, the network parameters of the feature extraction module can be kept unchanged, which can improve the training efficiency of the network model and thus reduce the training cost of the network model.

[0090] In an implementation manner of this embodiment, as Figure 4 shown, before training the second network model with the migration dataset to obtain the target network model, the method further includes:

[0091] Dividing the migration images in the migration dataset into several sub-datasets according to their respective corresponding class labels;

[0092] For every two sub-datasets in the several sub-datasets, generating a candidate dataset according to the first sub-dataset in the two sub-datasets and the second sub-dataset in the two sub-datasets;

[0093] Taking all the generated candidate datasets as the migration dataset.

[0094] Specifically, each migration image in each of the several sub-datasets is included in the migration dataset, and the class labels corresponding to the migration images in the several sub-datasets are the same. Thus, the class label corresponding to the migration image in each sub-dataset can be used as the class label corresponding to that sub-dataset, and then the class labels corresponding to each sub-dataset are different from each other. For example, the several sub-datasets include sub-dataset A and sub-dataset B. Sub-dataset A includes migration images a and migration image b, and sub-dataset B includes migration images c and migration image d. Then, the class label corresponding to migration image a is the same as the class label corresponding to migration image b, the class label corresponding to migration image c is the same as the class label corresponding to migration image d, and the migration label corresponding to migration image a is different from the migration label corresponding to migration image c, that is, the class label corresponding to sub-dataset A is different from the class label corresponding to sub-dataset B.

[0095] In an implementation manner of this embodiment, the several sub-datasets include at least two sub-datasets. The first sub-dataset and the second sub-dataset both include several sub-datasets, and the first sub-dataset is different from the second sub-dataset. A candidate dataset can be determined based on the first sub-dataset and the second sub-dataset. It can be understood that the number of candidate datasets can be determined according to the number of pairs of sub-datasets corresponding to the several sub-datasets. Among them, a pair of sub-datasets includes two different datasets, and each pair of sub-datasets is different from each other. For example, the several sub-datasets include sub-dataset A, sub-dataset B, and sub-dataset C. Then, the pairs of sub-datasets corresponding to the several sub-datasets include the pair of sub-datasets formed by sub-dataset A and sub-dataset B, the pair of sub-datasets formed by sub-dataset A and sub-dataset C, and the pair of sub-datasets formed by sub-dataset B and sub-dataset C.

[0096] In one implementation of this embodiment, the several sub-datasets may include one sub-dataset, denoted as the control sub-dataset. The first sub-dataset is composed of partial image regions of each migration image in the control sub-dataset, and the second sub-dataset is composed of partial image regions of each migration image in the control sub-dataset. It can be understood that each first migration image in the first sub-dataset corresponds one-to-one to each migration image in the control sub-dataset, and each first migration image is a partial image region of its corresponding migration image; each second migration image in the second sub-dataset corresponds one-to-one to each migration image in the control sub-dataset, and each second migration image is a partial image region of its corresponding migration image. For each migration image in the control sub-dataset, the first migration image corresponding to this migration image and the second migration image corresponding to this migration image do not overlap and together form this migration image. In a specific implementation, the process of determining the first sub-dataset and the second sub-dataset may be as follows: for each migration image in the control sub-dataset, obtain the foreground image and the background image of this migration image to obtain the foreground image and the background image corresponding to each migration image respectively; use the foreground image corresponding to each migration image as the first sub-dataset, and use the background image corresponding to each migration image as the second sub-dataset to obtain the first sub-dataset and the second sub-dataset. Of course, in practical applications, other methods may also be used to determine the first sub-dataset and the second sub-dataset. For example, for each migration image, randomly select a partial image region in this migration image as the first migration image, and use the part of this migration image excluding the first migration as the second migration, etc.

[0097] In one implementation of this embodiment, generating a candidate dataset according to the first sub-dataset in the two sub-datasets and the second sub-dataset in the two sub-datasets specifically includes:

[0098] For each migration image in the first sub-dataset and each migration image in the second sub-dataset, obtain the edge pixel points of this migration image, and select several target edge pixel points from all the obtained edge pixel points;

[0099] Select the first image corresponding to each target edge pixel point in this migration image;

[0100] Determine the second image corresponding to each first image according to this migration image;

[0101] Generate a candidate dataset according to all the first images and all the second images selected based on each migration image.

[0102] Specifically, the edge pixel points are the regional boundary pixel points of the object region carried by the migration image. For example, when the image to be denoised is an image carrying a red apple, the edge pixel points can be the boundary pixel points between the apple region and the non-apple region in the image to be denoised. In one implementation manner of this embodiment, the edge pixel points can be obtained by performing edge recognition on the migration image, where the edge recognition method can be the sobel edge extraction method.

[0103] Furthermore, the target pixel points are included in the obtained edge pixel points. Each target edge pixel point is included in its corresponding first image, and each target pixel point corresponds to one first image, and each first image corresponds to a second image. Thus, the number of first images is the same as the number of target pixel points, and the number of second images is the same as the number of first images. In addition, the target pixel point is any one of the edge pixel points. The first image can be based on a partial image region randomly selected in the edge image, and the second image is the image region of the migration image excluding the first image.

[0104] In one implementation manner of this embodiment, as Figure 4 shown, the determination process of the first image can be specifically as follows: taking the target edge pixel point as the center, selecting an image region with a first preset size in the migration image, and taking the selected image region as the first image. For example, the image size of the migration image is [w*h], and the first preset size is [w / 16, h / 16]. In addition, after the first image is selected, the unselected image region can be directly used as the second image, or a reference pixel point is selected in the migration image, taking the reference pixel point as the center, selecting an image region with a second preset size in the migration image, and taking the selected image region as the second image, where the reference pixel point is not included in the obtained edge pixel points. For example, the image size of the migration image is [w*h], and the first preset size is [w / 16, h / 16], etc.

[0105] In one implementation manner of this embodiment, training the second network model based on the migration dataset to obtain the target network model includes:

[0106] Selecting a candidate dataset from the migration dataset, and training the second network model based on the candidate dataset to obtain a candidate network model;

[0107] Obtaining the network parameters of the feature extraction module in the candidate network model, and configuring the network parameters in the second network model to update the second network model;

[0108] Steps of continuously selecting a candidate data set from the migration data set until all candidate data sets in the migration data set are selected, where each selected candidate data set is different from each other.

[0109] Specifically, the migration data set includes several candidate data sets. Each candidate data set is used as a training data set for the second network model. And before training the second network model based on the candidate data set, it can be determined whether there is a candidate network model obtained by training the second network model based on the candidate training set; when there is a candidate network model, obtain the model parameters of the candidate network model that have been trained by the candidate data set. If it has been trained by the candidate data set, the network parameters of the feature extraction module in the candidate network model are used to inherit the network parameters of the feature extraction module in the candidate network model, realizing transfer learning of network parameters. In this application, by dividing the migration data set into several candidate data sets and training the network parameters of the feature extraction module through several candidate data sets, the feature extraction module can be adapted to different task features, so that when determining the target image processing model later, even if the amount of training sample data is small, the model accuracy of the target image processing model can be guaranteed.

[0110] In an implementation manner of this embodiment, after training the second network model based on the migration data set to obtain a target network model and using the feature extraction module of the target network model as an image processing model, as Figure 2 shown, the method further includes:

[0111] Obtain a target migration data set and determine the third fully connected module corresponding to the target migration data set;

[0112] Obtain the image processing model and use the image processing model as the target feature extraction module corresponding to the target migration data set;

[0113] Cascade the target feature extraction module with the third fully connected module to obtain the third network model corresponding to the target migration data set;

[0114] Train the third network model based on the target migration data set to obtain the target image processing model corresponding to the target migration data set.

[0115] Specifically, the target migration data set is used to train the target image processing model. The target migration data set includes several target images. Each target image in the several target images is used as a training image and input into the third network model. The third network model trains the model parameters of other network structures in the third network model except the feature extraction module by learning the image features in each target image.

[0116] The model structure of the third network model is the same as that of the target image processing model, and both include a feature extraction module and a third fully connected module. Among them, the feature extraction module is the obtained image processing model and inherits the network parameters of this image processing model. It can be understood that the network parameters of the terminal feature extraction model of the third network model are trained network parameters. After inputting the target image in the target migration dataset into the feature extraction module of the third network model, the feature extraction module can accurately extract the image features of this target image.

[0117] Based on this, when training the third network model based on the target migration dataset, the network parameters of the feature extraction module in the third network model can be kept unchanged, and only the model parameters of other network structures in the third network model are trained. This can improve the training speed of the third network model, thereby improving the training efficiency of the target image processing model. Thus, when the target image processing model is obtained by training the third network model based on the target migration dataset, the network parameters of the feature extraction module in the target image processing model are the same as those of the feature extraction module in the third network model, and the model parameters of the third fully connected module of the target image processing model are different from those of the third fully connected module in the third network model. Among them, the model parameters of the third fully connected module in the third network model are the pre-configured initial model parameters, and the model parameters of the third fully connected module of the target image processing model are the model parameters after training based on the target migration dataset.

[0118] In summary, this embodiment provides a method, device, processing method and terminal device for generating an image processing model. The method includes obtaining an initial dataset and a migration dataset, and training a preset network model based on the initial dataset and the migration dataset to obtain a first network model; determining a second network model based on the feature extraction module, and generating a second dataset based on the migration dataset; training the second network model based on the second dataset to obtain a target network model, and using the feature extraction module of the target network model as the image processing model. This application utilizes the transferability of image visual features to learn the first network model from the initial dataset and the migration dataset, and trains the second network model that inherits the feature extraction module of the first network model using the migration dataset, and uses the feature extraction module of the trained target network model as the image processing model. In this way, for different image processing tasks, this image processing model can be directly used as the feature extraction module without training the feature extraction module, reducing the training cost of the image processing model.

[0119] Based on the above-mentioned image processing model generation method, this embodiment also provides a processing method, which is applied to the above-mentioned image processing model, as Figure 5 shown, the method includes:

[0120] Obtain the image to be processed;

[0121] Input the image to be processed into the image processing model, and output the output image corresponding to the image to be processed through the image processing model.

[0122] Based on the above-mentioned image processing model generation method, this embodiment also provides a generation device, as Figure 6 shown, the generation device includes:

[0123] An acquisition module 100, configured to acquire an initial data set and a migration data set, and train a preset network model based on the initial data set and the migration data set to obtain a first network model, where the preset network model includes a feature extraction module;

[0124] A determination module 200, configured to determine a second network model based on the feature extraction module, where the second network model includes the feature extraction module;

[0125] A training module 300, configured to train the second network model based on the migration data set to obtain a target network model, and use the feature extraction module of the target network model as the image processing model.

[0126] In one implementation, the initial data set includes a number of initial images, the migration images include a number of migration images, and the image processing tasks corresponding to the initial data set are different from the image processing tasks corresponding to the migration data set.

[0127] In one implementation, the acquisition module specifically includes:

[0128] A first acquisition unit, configured to acquire an initial data set and a migration data set;

[0129] A determination unit, configured to determine a training data set according to the initial data set and the migration data set;

[0130] A training unit, configured to train a preset network model based on the training data set to obtain a first network model.

[0131] In one implementation, the determination unit specifically includes:

[0132] An acquisition subunit, configured to acquire a number of preset angles;

[0133] An adjustment subunit, configured to, for each preset angle, rotate the initial images in the initial dataset and the migration images in the migration dataset according to the preset angle, respectively, to obtain a target initial dataset corresponding to the initial dataset and a target migration dataset corresponding to the migration dataset;

[0134] A determination subunit, configured to generate the training dataset according to the initial dataset, each target initial dataset, the migration dataset, and each target migration dataset.

[0135] In one implementation, the model parameters of the feature extraction module in the second network model are the same as the model parameters in the first network model.

[0136] In one implementation, the first network model includes a first fully connected module, and the determination module specifically includes:

[0137] A second acquisition unit, configured to acquire the feature extraction module in the first network model and determine a second fully connected module, where the output dimension of the second fully connected module is smaller than the output dimension of the first fully connected module;

[0138] A cascading unit, configured to cascade the feature extraction module and the second fully connected module to obtain a second network model.

[0139] In one implementation, the generating device further includes:

[0140] A partitioning module, configured to partition the migration images in the migration dataset into several sub-datasets according to their respective corresponding class labels, where the class labels corresponding to the migration images in the several sub-datasets are the same;

[0141] A generating module, configured to, for every two sub-datasets in the several sub-datasets, generate a candidate dataset according to a first sub-dataset in the two sub-datasets and a second sub-dataset in the two sub-datasets;

[0142] A setting module, configured to use all the generated candidate datasets as the migration dataset.

[0143] In one implementation, the generating module specifically includes:

[0144] A first selection unit, configured to, for each migration image in the first sub-dataset and the second sub-dataset, obtain the edge pixel points of the migration image, and select a plurality of target edge pixel points from the obtained edge pixel points; select, in the migration image, the first image corresponding to each of the target edge pixel points, where each target edge pixel point is a pixel point in the first image corresponding to it; determine, according to the migration image, the second image corresponding to each first image, where the second image is the image area of the migration image excluding the first image.

[0145] A generation unit, configured to generate a candidate dataset according to all the first images and all the second images selected based on each migration image.

[0146] In one implementation, the training module specifically includes:

[0147] A second selection unit, configured to select a candidate dataset from the migration dataset, and train the second network model based on the candidate dataset to obtain a candidate network model;

[0148] Obtain the network parameters of the feature extraction module in the candidate network model, and configure the network parameters in the second network model to update the second network model;

[0149] A loop unit, configured to continue the step of selecting a candidate dataset from the migration dataset until all the candidate datasets in the migration dataset have been selected, where the candidate datasets selected each time are different from each other.

[0150] In one implementation, the generation device further includes:

[0151] A third acquisition module, configured to acquire a target migration dataset and determine the third fully connected module corresponding to the target migration dataset;

[0152] A configuration module, configured to acquire the image processing model, and use the image processing model as the target feature extraction module corresponding to the target migration dataset;

[0153] A connection module, configured to cascade the target feature extraction module and the third fully connected module to obtain the third network model corresponding to the target migration dataset;

[0154] A learning module, configured to train the third network model based on the target migration dataset to obtain the target image processing model corresponding to the target migration dataset.

[0155] In one implementation, the network parameters of the target feature extraction module in the target image processing model are the same as the network parameters of the target feature extraction module in the third network model.

[0156] Based on the above image processing model generation method, this embodiment provides a computer-readable storage medium storing one or more programs that can be executed by one or more processors to implement the steps in the image processing model generation method as described in the above embodiment.

[0157] Based on the above image processing model generation method, this application also provides a terminal device, as Figure 7 shown, which includes at least one processor 20; a display screen 21; and a memory 22, and may further include a communication interface 23 and a bus 24. Among them, the processor 20, the display screen 21, the memory 22, and the communication interface 23 can complete mutual communication through the bus 24. The display screen 21 is set to display a user guidance interface preset in the initial setting mode. The communication interface 23 can transmit information. The processor 20 can call the logical instructions in the memory 22 to execute the method in the above embodiment.

[0158] In addition, when the logical instructions in the above memory 22 are implemented in the form of a software functional unit and sold or used as an independent product, they can be stored in a computer-readable storage medium.

[0159] The memory 22, as a computer-readable storage medium, can be set to store software programs and computer-executable programs, such as the program instructions or modules corresponding to the method in the embodiments of the present disclosure. The processor 20 executes functional applications and data processing by running the software programs, instructions, or modules stored in the memory 22, that is, implements the method in the above embodiment.

[0160] The memory 22 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the terminal device, etc. In addition, the memory 22 may include a high-speed random access memory and may also include a non-volatile memory. For example, various media that can store program codes such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs can also be transient storage media.

[0161] In addition, the specific processes of loading and executing multiple instructions by the above storage medium and the instruction processor in the terminal device have been described in detail in the above method and will not be repeated here.

[0162] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present application.

Claims

1. A method for generating an image processing model, characterized in that: The method comprises: Acquire an initial data set and a migration data set, and train a preset network model based on the initial data set and the migration data set to obtain a first network model, wherein the preset network model includes a feature extraction module; determining a second network model based on the feature extraction module, wherein the second network model includes the feature extraction module; Training the second network model based on the migration data set to obtain a target network model, and using the feature extraction module of the target network model as an image processing model; The initial dataset includes a plurality of initial images, the migration dataset includes a plurality of migration images, and the image processing task corresponding to the initial dataset is different from the image processing task corresponding to the migration dataset; Each initial image includes a category label corresponding to the initial image, and each transition image includes a category label corresponding to the transition image; The category is a category corresponding to the image processing task corresponding to the initial image or the migration image; The acquiring of the initial data set and the migration data set, and training the preset network model based on the initial data set and the migration data set to obtain the first network model specifically includes: Obtain the initial dataset and migration dataset; Determine a training dataset based on the initial dataset and the migration dataset; The preset network model is trained based on the training data set to obtain a first network model.

2. The method for generating an image processing model according to claim 1, wherein: The determining of the training dataset based on the initial dataset and the migration dataset specifically includes: Get several preset angles; For each preset angle, the initial image in the initial dataset and the migration image in the migration dataset are rotated according to the preset angle to obtain a target initial dataset corresponding to the initial dataset and a target migration dataset corresponding to the migration dataset; The training dataset is generated according to the initial dataset, each target initial dataset, the migration dataset and each target migration dataset.

3. The method for generating an image processing model according to claim 1, wherein: The model parameters of the feature extraction module in the second network model are the same as the model parameters in the first network model.

4. The method for generating an image processing model according to claim 3, wherein: The first network model includes a first fully connected module, and determining the second network model based on the feature extraction module specifically includes: Obtaining a feature extraction module in the first network model, and determining a second fully connected module, wherein an output dimension of the second fully connected module is smaller than an output dimension of the first fully connected module; The feature extraction module is cascaded with the second fully connected module to obtain a second network model.

5. The method for generating an image processing model according to claim 1, wherein: Before training the second network model based on the migration data set to obtain the target network model, the method further includes: Dividing the migration images in the migration dataset into a plurality of sub-datasets according to their corresponding category labels, wherein the category labels corresponding to the migration images in the plurality of sub-datasets are the same; For every two sub-datasets in the plurality of sub-datasets, generating a candidate data set according to a first sub-dataset in the two sub-datasets and a second sub-dataset in the two sub-datasets; All generated candidate datasets are used as migration datasets.

6. The method for generating an image processing model according to claim 5, wherein: Generating a candidate data set according to the first sub-data set of the two sub-data sets and the second sub-data set of the two sub-data sets specifically includes: For each transition image in the first subdataset and the second subdataset, edge pixels of the transition image are obtained, and a number of target edge pixels are selected from the obtained edge pixels; a first image corresponding to each target edge pixel is selected from the transition image, wherein each target edge pixel is a pixel in its corresponding first image; and a second image corresponding to each first image is determined based on the transition image, wherein the second image is the transition image minus the image region outside the first image. A candidate data set is generated according to all first images and all second images selected based on each migration image.

7. The method for generating an image processing model according to claim 5, wherein: The training of the second network model based on the migration data set to obtain a target network model includes: Selecting a candidate data set from the migration data set, and training the second network model based on the candidate data set to obtain a candidate network model; Obtaining network parameters of a feature extraction module in the candidate network model, and configuring the network parameters in the second network model to update the second network model; Continue the step of selecting a candidate dataset from the migration dataset until all the candidate datasets in the migration dataset are The candidate data sets are all selected, and the candidate data sets selected each time are different.

8. The method for generating an image processing model according to any one of claims 1 to 7, characterized in that: After training the second network model based on the migration data set to obtain a target network model, and using the feature extraction module of the target network model as an image processing model, the method further includes: Obtain a target migration dataset, and determine a third fully-connected module corresponding to the target migration dataset; Acquire the image processing model, and use the image processing model as a target feature extraction module corresponding to the target migration data set; Cascading the target feature extraction module and the third fully connected module to obtain a third network model corresponding to the target migration data set; The third network model is trained based on the target migration dataset to obtain a target image processing model corresponding to the target migration dataset.

9. The method for generating an image processing model according to claim 8, wherein: The network parameters of the target feature extraction module in the target image processing model are the same as the network parameters of the target feature extraction module in the third network model.

10. A processing method, characterized in that: The method is applied to the image processing model according to any one of claims 1 to 9, and the method comprises: Get the image to be processed; The image to be processed is input into the image processing model, and the image processing model outputs an output image corresponding to the image to be processed.

11. A generating device, characterized in that: The method for implementing the steps of the method for generating an image processing model according to any one of claims 1 to 9, or the method for implementing the steps of the processing method according to claim 10, wherein the generating device comprises: an acquisition module, configured to acquire an initial data set and a migration data set, and train a preset network model based on the initial data set and the migration data set to obtain a first network model, wherein the preset network model includes a feature extraction module; a determination module, configured to determine a second network model based on the feature extraction module, wherein the second network model includes the feature extraction module; A training module is used to train the second network model based on the migration data set to obtain a target network model, and use the feature extraction module of the target network model as an image processing model.

12. A computer-readable storage medium, characterized in that The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the method for generating an image processing model as described in any one of claims 1 to 9, or to implement the steps in the processing method as described in claim 10.

13. A terminal device, characterized in that: include: processor, memory, and communication bus; The memory stores a computer-readable program executable by the processor; The communication bus realizes the connection and communication between the processor and the memory; When the processor executes the computer-readable program, it implements the steps in the method for generating an image processing model as described in any one of claims 1 to 9, or implements the steps in the processing method as described in claim 10.

Images