Method for training deep learning network model, method for identifying item category, device, electronic equipment

Abstract

The application relates to the technical field of image recognition, and discloses a method for training a deep learning network model, which comprises the following steps: establishing an initial network model; and performing multiple times of migration training on the initial network model by using a preset article picture data set to obtain a preset deep learning network model. The preset deep learning network model is used to identify the article category of an article in a to-be-identified article picture. Since the preset deep learning network model is obtained by performing multiple times of migration training on the initial network model by using the preset article picture data set, the preset deep learning network model has higher precision and better classification accuracy. The application further discloses a method and device for identifying an article category and electronic equipment.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, for example to a method for training a deep learning network model, a method for identifying an item category, an apparatus, and an electronic device. BACKGROUND

[0002] At present, online shopping has become a daily habit of most users, and therefore the identification of item image content is also very important. By identifying the item category in the picture, we can search for the same item using the picture; by identifying the item category in the picture of the item purchased by the user, we can recommend the same item to the user. These applications are all based on a good-performing item category classifier, and if the classification accuracy of the item category classifier can be improved, the performance of the related application can be improved.

[0003] In the process of implementing the embodiments of the present disclosure, it is found that at least the following problems exist in the related art: For a classifier that identifies an item category through a deep learning network model, the classification accuracy is low because the current deep learning network model is usually trained using uniform data. SUMMARY

[0004] To provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. The summary is not an overall description of the application, nor is it intended to identify key / important components or delineate the scope of the embodiments, but to serve as a prelude to the detailed description below.

[0005] The embodiments of the present disclosure provide a method for training a deep learning network model, a method for identifying an item category, an apparatus, and an electronic device to solve the problem that the classification accuracy is low because the current deep learning network model is trained using uniform data.

[0006] In some embodiments, the method for training a deep learning network model comprises:

[0007] establishing an initial network model;

[0008] training the initial network model using a preset item picture dataset multiple times to obtain a preset deep learning network model.

[0009] In some embodiments, the method for identifying an item category comprises:

[0010] obtaining an item picture to be identified;

[0011] identifying the item picture to be identified using the preset deep learning network model;

[0012] obtaining an item category of an item in the item picture to be identified;

[0013] The preset deep learning network model is obtained by training the method for training a deep learning network model.

[0014] In some embodiments, the apparatus for identifying an item category comprises a processor and a memory storing program instructions, the processor being configured to execute the method for identifying an item category when executing the program instructions.

[0015] In some embodiments, the electronic device comprises the apparatus for identifying an item category.

[0016] The method for training a deep learning network model, the method for identifying an item category, the apparatus, and the electronic device provided by the embodiments of the present disclosure can achieve the following technical effects:

[0017] The preset deep learning network model is obtained by training the method for training a deep learning network model.

[0018] The foregoing general description and the following description are merely exemplary and explanatory, and are not intended to limit the present application. BRIEF DESCRIPTION OF DRAWINGS

[0019] One or more embodiments are exemplarily illustrated by corresponding drawings, which do not constitute a limitation on the embodiments, elements with the same reference numerals in the drawings are shown as similar elements, the drawings do not constitute a proportional limitation, and wherein:

[0020] Figure 1 is a schematic diagram of a method for training a network model provided by an embodiment of the present disclosure;

[0021] Figure 2 is a schematic diagram of another method for training a network model provided by an embodiment of the present disclosure;

[0022] Figure 3 is a schematic diagram of another method for training a deep learning network model provided by an embodiment of the present disclosure;

[0023] Figure 4 is a schematic diagram of a method for identifying an item category provided by an embodiment of the present disclosure;

[0024] Figure 5 is a schematic diagram of an apparatus for identifying an item category provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0025] In order to enable more detailed understanding of the features and technical contents of the embodiments of the present disclosure, the implementation of the embodiments of the present disclosure is described in detail below, and the attached drawings are only used for reference and do not limit the embodiments of the present disclosure. In the following technical description, in order to facilitate explanation, a plurality of details are provided to provide a full understanding of the disclosed embodiments. However, one or more embodiments can still be implemented without these details. In other cases, well-known structures and devices can be simplified to facilitate the drawings.

[0026] The terms "first", "second", and the like in the specification and claims of the embodiments of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present disclosure described herein can be implemented. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. Unless otherwise stated, the term "a plurality of" means two or more. In the embodiments of the present disclosure, the term "and / or" is a description of the association relationship of the objects, which means that there can be three relationships. For example, A and / or B means: A or B, or, A and B, the three relationships.

[0027] In combination Figure 1 As shown, the embodiments of the present disclosure provide a method for training a deep learning network model, comprising the following steps:

[0028] S11: Establish an initial network model.

[0029] The initial network model is an untrained network model established. The initial network model includes a convolutional layer, a pooling layer, a fully connected layer, and a classifier, wherein the convolutional layer is used to preliminarily extract the preliminary features of the items in the picture; the pooling layer is used to extract the main features of the items in the picture; the fully connected layer is used to summarize the features of the items in the picture; and the classifier is used to predict and identify the categories of the items. For example, ResNet101 is selected as the initial network model.

[0030] S12: Obtain a preset deep learning network model by using a preset item picture data set to perform multiple transfer training on the initial network model.

[0031] The preset item picture data set includes pictures containing items and category labels for labeling the attributes of the items in the pictures. Taking clothes as an example, collect model pictures wearing clothes or pictures containing only clothes, and perform corresponding category labeling on the attributes (such as collar type, sleeve type, clothing type, skirt type, or trouser type) of the clothes in the pictures to obtain corresponding category labels (such as long sleeve) to form an effective and reliable clothes picture data set.

[0032] The method for training a deep learning network model provided in the embodiment of the present disclosure is used to identify the item category of an item in a to-be-identified item picture by using a preset deep learning network model. Since the preset deep learning network model is obtained by performing multiple migration training on an initial network model by using a preset item picture dataset, the preset deep learning network model has higher precision and better classification accuracy.

[0033] Optionally, in combination with Figure 2 As shown, obtaining the preset deep learning network model by performing multiple migration training on the initial network model by using the preset item picture dataset includes the following steps:

[0034] S121: Dividing the preset item picture dataset into multiple preset item picture sub-datasets according to the similarity to the to-be-identified item picture.

[0035] The similarity to the to-be-identified item picture is the similarity degree of the picture size and the picture background color (i.e. the picture color excluding the item) of the to-be-identified item picture. Taking the item as clothes for example, when the to-be-identified clothes picture is a clothes picture containing only clothes taken in a real scene, it is considered that the similarity of the clothes picture crawled from the network to the to-be-identified clothes picture is low, and the similarity of the clothes picture containing only clothes obtained from a real application scene to the to-be-identified clothes picture is high; when the to-be-identified clothes picture is a clothes picture containing only clothes in the network, it is considered that the similarity of the clothes picture containing only clothes crawled from the network to the to-be-identified clothes picture is high, and the similarity of the clothes picture obtained from a real application scene to the to-be-identified clothes picture is low.

[0036] Optionally, the item pictures in the preset item picture dataset with a similarity less than a preset similarity are a first item picture sub-dataset; and / or, the item pictures in the preset item picture dataset with a similarity greater than or equal to the preset similarity are a second item picture sub-dataset.

[0037] The similarity of the item picture in the preset item picture dataset to the to-be-identified item picture can be obtained by the following formula:

[0038]

[0039] Wherein, S is the similarity of the item picture in the preset item picture dataset to the to-be-identified item picture, m1 is the picture size of the item picture in the preset item picture dataset, m0 is the picture size of the to-be-identified item picture, β is the weight value of the picture size similarity, c1 is the average chroma value of the background color of the item picture in the preset item picture dataset, c0 is the average chroma value of the background color of the to-be-identified item picture, and γ is the weight value of the picture background color size similarity.

[0040] The preset similarity can be determined by a person skilled in the art according to actual conditions, so as to divide the preset item picture dataset. The item pictures with a similarity less than the preset similarity in the preset item picture dataset are a first item picture sub-dataset, and the data value is small; and the item pictures with a similarity greater than or equal to the preset similarity in the preset item picture dataset are a second item picture sub-dataset, and the data value is large.

[0041] S122: training the initial network model according to a preset transfer training strategy by using the plurality of preset item picture sub-datasets.

[0042] S123: taking the trained initial network model as a preset deep learning network model.

[0043] The present disclosure divides data into low-value data (a first item picture sub-dataset) and high-value data (a second item picture sub-dataset) according to the similarity of the item pictures to be identified, trains the initial network model according to a preset transfer training strategy, and improves the accuracy of the model according to the accuracy stage, thereby improving the classification accuracy of the preset deep network model.

[0044] Optionally, in combination with Figure 3 As shown, training the initial network model according to a preset transfer training strategy by using the plurality of preset item picture sub-datasets includes the following steps:

[0045] S1221: training the initial network model by using the first item picture sub-dataset and obtaining an initial classification model.

[0046] S1222: training the initial classification model by using the second item picture sub-dataset and obtaining a deep learning classification model.

[0047] After obtaining the deep learning classification model, the deep learning classification model is taken as a preset learning network model. In the past, the deep learning method generally mixes all data together for training, and different levels of data have the same impact on the model, which is equivalent to wasting high-level data with high-precision annotation. The present disclosure divides data into several stages, first trains low-value data (a first item picture sub-dataset) with a large amount of data, and then gradually transfers to high-value data (a second item picture sub-dataset) with a small amount of data, so that the accuracy of the model is improved according to the accuracy stage, thereby improving the classification accuracy of the preset deep network model.

[0048] In some embodiments, the initial network model is trained according to the preset migration training strategy by using the plurality of preset item picture sub-datasets, and the method further comprises: after obtaining the deep learning classification model, obtaining a loss function according to a joint distribution distance of the first data of the first item picture sub-dataset and the second data of the second item picture sub-dataset; and optimizing and training the deep learning classification model by using the obtained loss function.

[0049] After the deep learning classification model is optimized and trained, the deep learning classification model after the optimization and training is taken as the preset learning network model. In this way, the plurality of preset item picture sub-datasets (the first item picture sub-dataset and the second item picture sub-dataset) can be migrated in the plurality of task-specific factor layers and the classification layer. By minimizing the joint distribution difference, a set of jointly adapted networks is formulated to match the joint distribution of these layers across domains, which can be effectively trained using backpropagation, further improving the accuracy of the deep network learning model.

[0050] Optionally, the loss function is obtained by the following formula:

[0051]

[0052] wherein Loss is the loss function, n s is the number of levels of the deep learning network model, is a cross-entropy loss function, and a is a balance parameter of the joint distribution difference, is a joint distribution distance of the first data and the second data, D s is a feature extracted from the first data by a convolutional layer L, D t is a feature extracted from the second data by the convolutional layer L.

[0053] The joint distribution distance of the first data and the second data is multiplied by a coefficient and a cross-entropy loss function to form a new loss function, and the deep learning network model is optimized and trained by using the new loss function, so that the distance of the joint distribution of the first data and the second data is minimized, which can improve the accuracy of the deep learning network model and further improve the classification accuracy of the preset deep learning network model.

[0054] In some embodiments, the preset deep learning network model is obtained by performing multiple migration training on the initial network model by using the preset item picture dataset, and the method further comprises: before performing the multiple migration training on the initial network model, performing an enhancement operation on the item pictures in the preset item picture dataset. For example, a random flipping operation, a random expansion and then random cutting operation, a random rotation operation, or a color value normalization operation.

[0055] Data augmentation is to increase the value of limited data without substantially increasing the data. In this way, the capacity of training data samples can be increased, and the generalization ability of the deep learning network model can be improved.

[0056] In some embodiments, in combination with Figure 4 As shown in the figure, the embodiment of the present disclosure provides a method for identifying the category of an article, comprising the following steps:

[0057] S41: obtaining an article picture to be identified.

[0058] S42: identifying the article picture to be identified by using a preset deep learning network model.

[0059] S43: obtaining the category of the article in the article picture to be identified.

[0060] The preset deep learning network model is obtained by training the aforementioned method for training the deep learning network model.

[0061] The method for identifying the category of an article provided by the embodiment of the present disclosure can be understood as an application method of the aforementioned method for training the deep learning network model. By using the method for identifying the category of an article provided by the embodiment of the present disclosure, the category of the article in the article picture to be identified is identified by using the preset deep learning network model. Since the preset deep learning network model is obtained by using the preset article picture data set to perform multiple migration training on the initial network model, the accuracy of the preset deep learning network model is higher, and the classification accuracy is also better.

[0062] In combination with Figure 5 As shown in the figure, the embodiment of the present disclosure provides an apparatus for identifying the category of an article, comprising a processor 50 and a memory 51, and can further comprise a communication interface 52 and a bus 53. The processor 50, the communication interface 52, and the memory 51 can communicate with each other through the bus 53. The communication interface 52 can be used for information transmission. The processor 50 can call the logical instructions in the memory 51 to execute the method for identifying the category of an article of the above-mentioned embodiments.

[0063] In addition, the logical instructions in the memory 51 described above can be implemented in the form of a software functional unit and sold or used as an independent product, which can be stored in a computer readable storage medium.

[0064] The memory 51 can be used to store software programs, computer executable programs, such as program instructions / modules corresponding to the method in the embodiments of the present disclosure. The processor 50 executes the function application and data processing by running the program instructions / modules stored in the memory 51, that is, implements the method for identifying the category of the article in the above-mentioned method embodiments.

[0065] The memory 51 can include a program storage area and a data storage area, wherein the program storage area can store an operating system and application programs required by at least one function; the data storage area can store data created according to the use of the terminal device, etc. In addition, the memory 51 can include a high-speed random access memory, and can also include a non-volatile memory.

[0066] The embodiments of the present disclosure provide an electronic device (for example: computer, mobile phone, etc.), which includes the above-mentioned device for identifying the category of the article.

[0067] The embodiments of the present disclosure provide a computer readable storage medium, which stores computer executable instructions, and the computer executable instructions are configured to execute the above-mentioned method for identifying the category of the article.

[0068] The embodiments of the present disclosure provide a computer program product, which includes a computer program stored on a computer readable storage medium, and the computer program includes program instructions, when the program instructions are executed by a computer, the computer executes the above-mentioned method for identifying the category of the article.

[0069] The above-mentioned computer readable storage medium can be a transitory computer readable storage medium, or a non-transitory computer readable storage medium.

[0070] The technical solutions of the embodiments of the present disclosure can be embodied in the form of a software product, and the computer software product is stored in a storage medium, and includes one or more instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in the embodiments of the present disclosure. The above-mentioned storage medium can be a non-transitory storage medium, including: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, etc. various media that can store program codes, or a transitory storage medium.

[0071] The above description and drawings are illustrative of embodiments of the present disclosure and are not intended to be limiting. Other embodiments can include structural, logical, electrical, process, and other changes. Embodiments are illustrative of the many possible variations that are readily undertaken. Individual components and functions are optional unless explicitly required, and the order of operations can be varied. Portions and features of some embodiments can be included in, or substituted for, those of other embodiments. The scope of the present disclosure encompasses the entire scope of the following claims, and all available equivalents of the claims. When used in this application, the terms "first," "second," and the like, do not denote any order, quantity, or importance, but rather are used to distinguish one element from another. For example, a first element could be termed a second element, and, similarly, a second element could be termed a first element, without changing the meaning of the description, so long as all occurrences of the "first element" are renamed consistently and all occurrences of the "second element" are renamed consistently. The first element and the second element are both elements, but they are not necessarily the same element. Also, the phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting. As used in the description of the embodiments and the claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. Similarly, the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. In addition, the term "comprises / comprising" and / or "comprises / comprising," when used in this application, refers to the presence of stated features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. Without more limitations, an element preceded by "comprises a" does not, without more limitations, preclude the existence of additional identical elements in the process, method, article, or apparatus including the element. In this document, each embodiment is highlighted by the differences from other embodiments. Identical or similar parts between embodiments can be mutually referred to. For the method, product, etc. disclosed by the embodiments, if it corresponds to the method part disclosed by the embodiments, the relevant part can be referred to the description of the method part.

[0072] Those skilled in the art can understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods for each specific application to realize the described functions, but such implementation should not be considered beyond the scope of the embodiments of the present disclosure. The skilled person can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0073] In the embodiments disclosed herein, the disclosed methods, products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the above-described device embodiments are only schematic, for example, the division of the units can only be a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms. The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to implement the embodiments. In addition, each functional unit in the embodiments of the present disclosure can be integrated in one processing unit, or each unit can be a physically independent unit, or two or more units can be integrated in one unit.

[0074] The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other processing device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other processing device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

Claims

1. A method for training a deep learning network model, the method comprising: The method comprises: establishing an initial network model; dividing a preset item picture dataset into a plurality of preset item picture sub-datasets according to similarity with a picture to be identified; training the initial network model according to a preset transfer training strategy by using the plurality of preset item picture sub-datasets; taking the trained initial network model as a preset deep learning network model; wherein, the item pictures in the preset item picture dataset with similarity less than a preset similarity are a first item picture sub-dataset, and the item pictures in the preset item picture dataset with similarity greater than or equal to the preset similarity are a second item picture sub-dataset; the training of the initial network model according to the preset transfer training strategy by using the plurality of preset item picture sub-datasets comprises: performing initial training on the initial network model by using the first item picture sub-dataset and obtaining an initial classification model; performing secondary training on the initial classification model by using the second item picture sub-dataset and obtaining a deep learning classification model; obtaining a loss function according to a joint distribution distance of first data of the first item picture sub-dataset and second data of the second item picture sub-dataset; and performing optimization training on the deep learning classification model by using the obtained loss function.

2. The method of claim 1, wherein, The initial network model comprises a convolutional layer, a fully connected layer and a classifier, wherein the convolutional layer is used for preliminary extraction of preliminary features of an item in a picture; the fully connected layer is used for summarizing various features of the item in the picture; and the classifier is used for predicting and identifying the category of the item.

3. The method of claim 1, wherein, The preset item picture dataset comprises pictures containing items and category labels for labeling attributes of the items in the pictures.

4. The method of claim 1, wherein, The similarity with the picture to be identified is the similarity of the picture size and the background color of the picture to be identified.

5. The method of claim 1, wherein, The similarity of the item picture in the preset item picture data set and the item picture to be identified can be obtained by the following formula: Wherein, S is the similarity of the item picture in the preset item picture data set and the item picture to be identified, m1 is the picture size of the item picture in the preset item picture data set, m0 is the picture size of the item picture to be identified, β is the weight value of the picture size similarity, c1 is the average chroma value of the background color of the item picture in the preset item picture data set, c0 is the average chroma value of the background color of the item picture to be identified, and γ is the weight value of the picture background color size similarity.

6. The method of claim 1, wherein, The loss function is obtained by the following formula: wherein, Loss is a loss function, is a number of levels of the deep learning network model, is a cross-entropy loss function, and α is a balance parameter of the joint distribution difference, is a joint distribution distance of the first data and the second data, is a feature extracted from the first data by a convolutional layer L, is a feature extracted from the second data by the convolutional layer L.

7. The method according to any one of claims 1 to 6, characterized in that, The preset deep learning network model is obtained by performing multiple transfer training on the initial network model by using the preset item picture dataset, and further comprises: Before performing multiple transfer training on the initial network model, performing enhancement operation on the item pictures in the preset item picture dataset.

8. A method for identifying a class of an article, characterized by, The method comprises: obtaining a picture to be identified; identifying the picture to be identified by using a preset deep learning network model; obtaining the category of an item in the picture to be identified; wherein, the preset deep learning network model is obtained by training according to the method for training a deep learning network model according to any one of claims 1 to 7.

9. An apparatus for identifying a class of an item, comprising a processor and a memory having stored program instructions, wherein, The processor is configured to execute the method for identifying the category of an item according to claim 8 when executing the program instructions.

10. An electronic device, comprising: The device for identifying the category of an item according to claim 9. The device for identifying the category of an item according to claim 9.

Images