Network model training method, image attribute recognition method, device and electronic equipment

Abstract

A network model training method, an image attribute recognition method, an image attribute recognition device (500) and an electronic device (600). The network model training method comprises: acquiring an image sample set (101); constructing a basic model and a target loss function corresponding to the basic model (102); training the basic model according to the image sample set and the loss function until the basic model converges; and taking the converged basic model as a recognition model for recognizing image attributes (105).

Description

Technical Field

[0001] The embodiments of the present application relate to computer technology, and in particular to a network model training method, an image attribute recognition method, a device, and an electronic device. Background Art

[0002] Image recognition, the use of computers to process, analyze, and understand images to identify various patterns of objects, is a practical application of deep learning algorithms. Current image recognition methods directly feed images into convolutional neural networks for feature extraction. These extracted features are then processed in the convolutional neural network's fully connected layers to produce the final image prediction.

[0003] However, the image recognition results obtained through such image recognition will ignore many image attributes and the correlation and sequence between image attributes. For example, when recognizing a human body image, if the human body image contains a hat, glasses, a top, a bag, a bottom, shoes, etc., there will be correlation and sequence between the top and the bottom. If directly recognized by the above-mentioned existing technology, the correlation and sequence between the top and the bottom will be ignored, resulting in inaccurate recognition results. Summary of the Invention

[0004] The present application provides a network model training method, an image attribute recognition method, a device and an electronic device, which can accurately identify image attributes and the correlation between the attributes.

[0005] In a first aspect, an embodiment of the present application provides a network model training method, the method comprising:

[0006] Acquire an image sample set, wherein the image sample set includes a plurality of image attribute initial values;

[0007] Constructing a basic model and a target loss function corresponding to the basic model, wherein the basic model includes a convolutional neural network model and a recurrent neural network model;

[0008] Inputting the image sample set into the basic model to perform image attribute recognition to obtain a first training result obtained according to the recurrent neural network model and a second training result obtained according to the convolutional neural network model;

[0009] Jointly training the convolutional neural network model and the recurrent neural network model according to the first training result, the second training result, the initial value of the image attribute, and the target loss function until the basic model converges;

[0010] The converged basic model is used as a recognition model for recognizing image attributes.

[0011] In a second aspect, an embodiment of the present application further provides an image attribute recognition method, which includes:

[0012] receiving an image attribute recognition request;

[0013] Acquire an image to be identified according to the image attribute identification request;

[0014] Call the pre-trained image attribute recognition model;

[0015] Inputting the image to be identified into a pre-trained image attribute recognition model to identify the image attributes of the image to be identified to obtain an image attribute recognition result;

[0016] The image attribute recognition model is trained using the network model training method provided in the embodiment of the present application.

[0017] In a third aspect, an embodiment of the present application provides a network model training device, comprising:

[0018] A first acquisition module is used to acquire an image sample set, wherein the image sample set includes a plurality of initial values ​​of image attributes;

[0019] A construction module is used to construct a basic model and a target loss function corresponding to the basic model, wherein the basic model includes a convolutional neural network model and a recurrent neural network model;

[0020] a first recognition module, configured to input the image sample set into the basic model to perform image attribute recognition, so as to obtain a first training result obtained according to the recurrent neural network model and a second training result obtained according to the convolutional neural network model;

[0021] a training module, configured to jointly train the convolutional neural network model and the recurrent neural network model according to the first training result, the second training result, the initial value of the image attribute, and the target loss function until the basic model converges;

[0022] A determination module is used to use the converged basic model as a recognition model for recognizing image attributes.

[0023] In a fourth aspect, an embodiment of the present application provides an image attribute recognition device, comprising:

[0024] A receiving module, configured to receive an image attribute recognition request;

[0025] A second acquisition module is used to acquire the image to be identified according to the image attribute recognition request;

[0026] The calling module is used to call the pre-trained image attribute recognition model;

[0027] a second recognition module, configured to input the image to be recognized into a pre-trained image attribute recognition model, recognize the image attributes of the image to be recognized, and obtain an image attribute recognition result;

[0028] The image attribute recognition model is an image attribute recognition model obtained by the network model training method provided in the embodiment of the present application.

[0029] In a fifth aspect, an embodiment of the present application provides a storage medium on which a computer program is stored, wherein, when the computer program is executed on a computer, the computer executes the network model training method or image attribute recognition method provided in this embodiment.

[0030] In a sixth aspect, an embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor calls the computer program stored in the memory to execute:

[0031] Acquire an image sample set, wherein the image sample set includes a plurality of image attribute initial values;

[0032] Constructing a basic model and a target loss function corresponding to the basic model, wherein the basic model includes a convolutional neural network model and a recurrent neural network model;

[0033] Inputting the image sample set into the basic model to perform image attribute recognition to obtain a first training result obtained according to the recurrent neural network model and a second training result obtained according to the convolutional neural network model;

[0034] Jointly training the convolutional neural network model and the recurrent neural network model according to the first training result, the second training result, the initial value of the image attribute, and the target loss function until the basic model converges;

[0035] The converged basic model is used as a recognition model for recognizing image attributes.

[0036] In a seventh aspect, an embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor calls the computer program stored in the memory to execute:

[0037] receiving an image attribute recognition request;

[0038] Acquire an image to be identified according to the image attribute identification request;

[0039] Call the pre-trained image attribute recognition model;

[0040] input the image to be identified into a pre-trained image attribute recognition model, recognize the image attribute of the image to be identified to obtain an image attribute recognition result;

[0041] The image attribute recognition model is obtained by training the network model according to the training method provided in the embodiments of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0042] The technical solutions of the present application and the beneficial effects thereof will become apparent through the following detailed description of the specific embodiments of the present application in conjunction with the accompanying drawings.

[0043] Figure 1 is a first flowchart of a network model training method provided in the embodiments of the present application.

[0044] Figure 2 is a second flowchart of a network model training method provided in the embodiments of the present application.

[0045] Figure 3 is a flowchart of an image attribute recognition method provided in the embodiments of the present application.

[0046] Figure 4 is a structural diagram of a network model training device provided in the embodiments of the present application.

[0047] Figure 5 is a structural diagram of an image attribute recognition device provided in the embodiments of the present application.

[0048] Figure 6 is a first structural diagram of an electronic device provided in the embodiments of the present application.

[0049] Figure 7 is a second structural diagram of an electronic device provided in the embodiments of the present application. DETAILED DESCRIPTION

[0050] Please refer to the drawings, wherein the same component symbols represent the same components, and the principles of the present application are exemplified in an appropriate operating environment. The following description is based on the exemplified embodiments of the present application, which should not be regarded as limiting other embodiments of the present application not described in detail.

[0051] Please refer to Figure 1 , Figure 1 is a first flowchart of a network model training method provided in the embodiments of the present application. The flowchart of the network model training method can include:

[0052] 101、obtain an image sample set, the image sample set includes multiple images, such as human body images, animal images, plant images, etc., in the embodiment of the application, multiple human body images in the image sample set can be taken as training images of the network model.

[0053] For example, randomly download pictures from a network gallery, filter out pictures containing human body images to form an image sample set. Or select multiple human body images as an image sample set according to the direction of model training, for example, the direction of model training is to find the relevance between clothes in human body images, and the images in the image sample set can be selected to contain multiple different dressing combinations of human body images.

[0054] It should be noted that in order to ensure that the vector dimension input in the neural network remains fixed and the neural network does not change dynamically, during the process of obtaining human body images, different human body images can be cropped so that the size of each cropped human body image is the same, thereby achieving the purpose of network model training.

[0055] It can be understood that according to the direction of network model training, the relevance between facial features or the relevance between limbs can also be selected as the direction of model training. At this time, in the process of obtaining the image sample set, the images containing facial features in multiple human body images can be cropped into different images of the same size, for example, obtaining a preset number of feature points of the face in the human body image, if the acquisition is completed, the image containing the facial features in the human body image is disassembled, otherwise, if the number of feature points obtained from the human body image does not reach the preset number of feature points, it is considered that the human body image cannot be used as an image sample set.

[0056] It should be noted that after obtaining the image sample set, each training image in the image sample set has its corresponding initial value of image attribute, for example, the human body image contains attributes such as upper garment, lower garment, hat, shoe, etc., and the relevance between the upper garment and the lower garment is 10%, then the relevance between the upper garment and the lower garment 10% can be used as the initial value of the image attribute, the image attribute initial value can be multiple or one, which needs to be determined according to the number of attributes in the training image and the correlation between the attributes.

[0057] 102、construct a basic model and a target loss function corresponding to the basic model.

[0058] The basic model can be created by different types of network models, for example, a convolutional neural network model (Convolutional Neural Networks, CNN) and a recurrent neural network (Recurrent Neural Network, RNN) model can be used to create the basic model.

[0059] In some embodiments, an input layer can be set, which is used to input training images in the image sample set into the basic model, and then the input layer is connected to the convolution layer, the convolution layer is connected to the pooling layer and the recurrent neural network respectively, the recurrent neural network is connected to the first fully connected layer, the pooling layer is connected to the second fully connected layer, and the first fully connected layer and the second fully connected layer serve as the output layer of the basic model.

[0060] It should be noted that the input layer, convolution layer, pooling layer and the second fully connected layer are connected in sequence to form a convolutional neural network, and the recurrent neural network is set between the convolution layer and the first fully connected layer.

[0061] In some implementations, the training sample input is a continuous sequence of varying lengths, such as a time-based sequence: a continuous speech segment or a continuous handwritten text segment. In this case, a recurrent neural network can handle the uncertainty of such input training values. However, since recurrent neural networks also suffer from the vanishing gradient problem, they struggle to process long sequences of data. Therefore, a recurrent neural network can employ a gated recurrent unit (GRU) network. The recurrent unit in a GRU network contains only two gates: an update gate and a reset gate. These gates do not form self-loops, but instead recur directly between system states.

[0062] For the basic model formed by the combination of the convolutional neural network and the gated recurrent unit network, the gated recurrent unit network corresponds to the first loss function, and the convolutional neural network corresponds to the second loss function. The first loss function and the second loss function can be different types of loss functions or the same type of loss functions. The target loss function corresponding to the basic model is obtained by the first loss function and the second loss function.

[0063] In some embodiments, the first loss function can be multiplied by the loss coefficient and then added to the second loss function to obtain the objective function corresponding to the basic model, wherein the loss coefficient can be a parameter obtained through experiments and can be set between 0.8 and 1.

[0064] 103. Input the image sample set into the basic model to perform image attribute recognition to obtain a first training result obtained according to the recurrent neural network model and a second training result obtained according to the convolutional neural network model.

[0065] A training image in an image sample set is input into the input layer, where the training image may be a human body image, and the attributes of the image are identified through the constructed basic model and the target function corresponding to the basic model.

[0066] Among them, the training image is first calculated through the convolution layer to obtain the first eigenvalue, and the first eigenvalue output by the last layer of the convolution layer is then input into the pooling layer and the recurrent neural network respectively. The second eigenvalue output by the recurrent neural network is input into the first fully connected layer to obtain the first training result, and the third eigenvalue input by the pooling layer is input into the second fully connected layer to obtain the second training result.

[0067] It should be noted that the first and second training results are not identical. The final training result obtained by the base model is derived from the first and second training results. For example, the first and second training results can be intersected, and the target training result in the intersection is the final training result obtained by the base model.

[0068] In some implementations, the first training result and the second training result may be added together to obtain a final training result, or part of the training results may be selected from the first training result and the second training result as the final training result according to a preset rule.

[0069] 104. Jointly train the convolutional neural network model and the recurrent neural network model according to the first training result, the second training result, the initial value of the image attribute, and the target loss function until the basic model converges.

[0070] In some embodiments, by inputting different training images, multiple first training results and second training results are obtained according to the basic model, and then the final training result is obtained based on the first training results and the second training results. The target loss value can be obtained by inputting the final training result and the initial value of the image attribute into the target loss function.

[0071] For example, the input training image is a human body image, in which the person in the human body image wears multiple items of clothing, such as tops, bottoms, shoes, and hats. Each item of clothing can be regarded as one of the attributes of the human body image. There is a specific correlation and a correlation degree corresponding to the correlation between the attributes. The correlation degree can be regarded as the initial value of the image attribute.

[0072] In some embodiments, after obtaining the final training results and the initial values ​​of the image attributes, the two are input into the target loss function to obtain the corresponding target loss value. It can be judged whether the target loss value is close to the preset loss value. If the target loss value is within a preset range from the preset loss value, it is considered that the basic model has been trained and is in a convergence state.

[0073] For example, an electronic device inputs a human image into a basic model, outputs a first training result in the first fully connected layer, and inputs a second training result in the second fully connected layer. The result output by the first fully connected layer includes the correlations between attributes. For example, the first training result includes the attributes of a padded jacket, gloves, and shorts. There is a correlation between the padded jacket and gloves, but not between the padded jacket and shorts. The second training result includes the attributes of a padded jacket, gloves, and a short skirt. The final training result can be taken as the intersection of the first and second training results. This final training result then includes the attributes of the padded jacket and gloves, as well as the correlation between the two attributes.

[0074] The final training result is input into the target loss function to obtain the target loss value. The target loss value is compared with the preset loss value to determine whether the target loss value meets the preset conditions. For example, the preset range of the preset loss value is 1-10. When the preset loss value is 80 and the target loss value is 75, it can be determined that the target loss value is within the preset range. The target loss value meets the preset conditions and it is considered that the basic model has converged.

[0075] In some embodiments, the first training result and the initial value of the image attribute can be directly input into the first loss function, and the second training result and the initial value of the image attribute can be input into the second loss function. Since the target loss function is obtained based on the first loss function and the second loss function, the target loss value can be directly calculated using the loss function when the first training result and the second training result are obtained. The target loss value is compared with the preset loss value to determine whether the basic model has converged. For example, when the target loss value is less than or equal to the preset loss value, it is considered that the target loss value meets the preset condition, and the basic model is considered to have converged.

[0076] It should be noted that in the training of the basic model, the number of training times can be multiple. Through the joint training of the convolutional neural network model and the recurrent neural network model, the basic model is finally converged to achieve the expected effect of the basic model training.

[0077] 105. The converged basic model is used as a recognition model for identifying image attributes.

[0078] Among them, the converged basic model is used as an image attribute recognition model for image attribute recognition. The image attribute recognition model can be applied to electronic devices to recognize the image attributes stored by the user in the electronic device according to the image attribute recognition model. The correlation between image attributes can be obtained according to the image attribute recognition results, and the accuracy of image attribute recognition can also be improved.

[0079] From the above, it can be seen that the network model training method provided in the embodiment of the present application obtains an image sample set, which includes multiple image attribute initial values; constructs a basic model and a target loss function corresponding to the basic model, and the basic model includes a convolutional neural network model and a recurrent neural network model; inputs the image sample set into the basic model for image attribute recognition to obtain a first training result obtained according to the recurrent neural network model and a second training result obtained according to the convolutional neural network model; jointly trains the convolutional neural network model and the recurrent neural network model according to the first training result, the second training result, the image attribute initial values ​​and the target loss function until the basic model converges; and uses the converged basic model as a recognition model for identifying image attributes.

[0080] The image attribute recognition model obtained in this way can improve the accuracy of image attribute recognition and can also identify the correlation between image attributes.

[0081] See also Figure 2 , Figure 2 : This is a second flow chart of the network model training method provided in an embodiment of the present application. The network model training method may include:

[0082] 201. Obtain an image sample set.

[0083] The image sample set includes a variety of images, such as human images, animal images, plant images, etc. In an embodiment of the present application, multiple human images in the image sample set can be used as training images for the network model.

[0084] In some embodiments, multiple human images are used as training images, and the attributes of each human image and the correlation between the attributes can be extracted. For example, in a human image, there are image attributes such as hats, glasses, tops, bottoms, and shoes. The correlation between different attributes is different. For example, there is no close correlation between wearing glasses and wearing tops, and there is no close correlation between wearing shoes and wearing hats. At this time, attributes with correlations can be obtained, such as the correlation between shoes and bottoms, and between tops and bottoms, as well as the corresponding correlation between the associated attributes. Specifically, sports shoes, sports pants, and sports tops are correlated, and the three also have a correlation. The image attributes and the corresponding correlation between the image attributes can be used as an initial value of an image attribute.

[0085] 202. Build a basic model based on convolutional neural networks and recurrent neural networks.

[0086] Among them, an input layer can be set first, which is used to input training images, and then a convolution layer can be set. The convolution layer is used to perform preliminary image feature extraction on the input training images to obtain the first eigenvalue, and then the first eigenvalue is input into the next layer of basic model structure.

[0087] The convolutional layer is connected to the recurrent neural network. The recurrent neural network can process the first eigenvalue output by the convolutional layer and input the second eigenvalue. The other side of the recurrent neural network is connected to the first fully connected layer. The first fully connected layer can serve as an output layer of the basic model, used to process the second eigenvalue and output the first training result.

[0088] The input layer, convolutional layer, pooling layer, and second fully connected layer are sequentially connected to form a convolutional neural network. The first eigenvalue output by the convolutional layer is processed by the pooling layer to obtain the third eigenvalue. Finally, the second fully connected layer processes the third eigenvalue and outputs the second training result. The entire basic model can be considered as a combination of a convolutional neural network model and a recurrent neural network model.

[0089] In some embodiments, multiple convolutional layers may be provided, for example, convolutional layers including conv3, conv6, and conv9. After the input human body image is processed by multiple convolutional layers, a feature map of a certain dimension is obtained, and the feature map is used as the first eigenvalue. The recurrent neural network may be a gated recurrent unit network, i.e., a GRU neural network.

[0090] 203. Construct a first loss function of the recurrent neural network and a second loss function corresponding to the convolutional neural network.

[0091] The first loss function and the second loss function may be loss functions of the same type or loss functions of different types. For example, the first loss function and the second loss function may both be cross entropy loss functions.

[0092] For example, the output of the last layer of a convolutional neural network can be processed using the softmax algorithm. This step usually calculates the probability that the output belongs to a certain class. For a single sample, the output is a vector. The softmax formula is:

[0093]

[0094] Then make a cross entropy between the output vector after softmax and the actual label of the sample. The formula is as follows:

[0095] H y (y)=-∑ i y′ i log(y i );

[0096] y′ i Represents the value of the i-th label in the actual label, y i It is the i-th element in the softmax output vector [Y1,Y2,Y3...].

[0097] 204. Construct a target loss function corresponding to the basic model according to the first loss function and the second loss function.

[0098] In some embodiments, the first loss function can be multiplied by the loss coefficient and then added to the second loss function to obtain the objective function corresponding to the basic model, wherein the loss coefficient can be a parameter obtained through experiments and can be set between 0.8 and 1.

[0099] It is understandable that according to the training direction of the basic model, the setting of the target loss function also needs to be adjusted according to the second training direction. Setting the loss coefficient before the first loss function and multiplying it with the first loss function is a way to adjust the target function.

[0100] 205. Input the image sample set into the basic model to perform image attribute recognition to obtain a first training result obtained according to the recurrent neural network model and a second training result obtained according to the convolutional neural network model.

[0101] A training image in an image sample set is input into the input layer, where the training image may be a human body image, and the attributes of the image are identified through the constructed basic model and the target function corresponding to the basic model.

[0102] Among them, the training image is first calculated through the convolution layer to obtain the first eigenvalue, and the first eigenvalue output by the last layer of the convolution layer is then input into the pooling layer and the recurrent neural network respectively. The second eigenvalue output by the recurrent neural network is input into the first fully connected layer to obtain the first training result, and the third eigenvalue input by the pooling layer is input into the second fully connected layer to obtain the second training result.

[0103] It should be noted that the first and second training results are not identical. The final training result obtained by the base model is derived from the first and second training results. For example, the first and second training results can be intersected, and the target training result in the intersection is the final training result obtained by the base model.

[0104] In some implementations, the first training result and the second training result may be added together to obtain a final training result, or part of the training results may be selected from the first training result and the second training result as the final training result according to a preset rule.

[0105] 206. Input the first training result, the second training result, and the initial value of the image attribute into the target loss function to obtain a target loss value.

[0106] In some embodiments, by inputting different training images, multiple first training results and second training results are obtained according to the basic model, and then the final training result is obtained based on the first training results and the second training results. The target loss value can be obtained by inputting the final training result and the initial value of the image attribute into the target loss function.

[0107] In some embodiments, the first training result and the initial values ​​of the image attributes can be directly input into the first loss function, and the second training result and the initial values ​​of the image attributes can be input into the second loss function. Since the target loss function is derived from the first and second loss functions, the target loss value can be directly calculated using the loss function when the first and second training results are obtained. The target loss value is compared with the preset loss value to determine whether the basic model has converged. For example, when the target loss value is less than or equal to the preset loss value, it is considered that the target loss value has met the preset condition, and the basic model is considered to have converged.

[0108] 207. Adjust the parameters of the basic model according to the target loss value until the basic model converges.

[0109] In some embodiments, when the target loss value does not meet the preset conditions, for example, the target loss value is not within the preset range, or the target loss value does not reach the preset loss value, it can be considered that the basic model training is not completed, and the training results output by the basic model cannot achieve the expected results, so the model parameters of the basic model need to be adjusted.

[0110] In some embodiments, since the base model is established based on the convolutional neural network model and the recurrent neural network model, when adjusting the parameters of the base model, some model parameters in the convolutional neural network model and the recurrent neural network model may be adjusted. The model parameters may be adjusted using a backpropagation algorithm.

[0111] 208. The converged basic model is used as a recognition model for identifying image attributes.

[0112] Among them, the converged basic model is used as an image attribute recognition model for image attribute recognition. The image attribute recognition model can be applied to electronic devices to recognize the image attributes stored by the user in the electronic device according to the image attribute recognition model. The correlation between image attributes can be obtained according to the image attribute recognition results, and the accuracy of image attribute recognition can also be improved.

[0113] For example, if a random human image is input into a converged basic model, it can accurately identify the type of clothing worn in the image and the relationships between the clothing items; or accurately identify the facial features of a person and the relationships between them. This indicates that the converged basic model has accurately identified the attributes of the input image and the relationships between the attributes, and can serve as an image attribute recognition model.

[0114] In summary, the network model training method provided in the embodiment of the present application obtains an image sample set and the initial values ​​of image attributes included in the image sample set, then constructs a basic model based on a convolutional neural network and a recurrent neural network, sets a first loss function for the recurrent neural network, and sets a second loss function for the convolutional neural network, and obtains a target loss function based on the first loss function and the second loss function; the image sample set is input into the basic model to train the basic model to obtain a first training result and a second training result, and finally, the parameters of the basic model are adjusted according to the initial values ​​of the image attributes, the target loss function, the first training result and the second training result until the basic model converges, and the converged model is used as an image attribute recognition model to accurately identify image attributes and the correlation between attributes.

[0115] See also Figure 3 , Figure 3 : is a flow chart of the image attribute recognition method provided in an embodiment of the present application. The image attribute recognition method may include the following process:

[0116] 301. Receive an image attribute recognition request.

[0117] The image attribute recognition request can be triggered by the electronic device receiving a touch operation, voice operation, or receiving an instruction to start the corresponding target application. In addition, the image attribute recognition request can be automatically triggered at intervals of preset duration or based on certain triggering rules. For example, when it is detected that the current display interface of the electronic device includes multiple images, such as when it is detected that the electronic device starts a browser application to browse an article page containing images, it can automatically trigger the generation of an image attribute recognition request, and perform image attribute recognition on multiple images according to the image attribute recognition model. This allows the electronic device to accurately identify the image attributes and the correlation between the attributes.

[0118] In some embodiments, when a user uses an electronic device for online shopping, a variety of clothing or accessories can be input into a virtual human image, where tops, bottoms, shoes, hats, earrings, necklaces, etc. can all be attribute information of the human body image. The user can input these items into the virtual human body image and wear them in the corresponding positions to obtain a new virtual human body image. After the user input is completed, the user can choose to recognize, and the electronic device receives the image attribute recognition request to recognize the new virtual human body image.

[0119] 302、obtaining the image to be identified according to the image attribute identification request.

[0120] In some embodiments, when the interface browsed by the user contains multiple images, the user can click a specific position or divide a region with a finger on the electronic device to select the image to be identified, and the electronic device can obtain the image to be identified according to the selection position or region of the user.

[0121] In some embodiments, after the user inputs the image attribute identification request, the electronic device can actively obtain the image to be identified according to the image attribute identification request, for example, when the user browses pictures, the electronic device can actively search for the picture to be identified according to the image attribute identification request.

[0122] 303、identifying the target subject in the image to be identified according to the image attribute identification request.

[0123] In some embodiments, the image attribute identification request contains the specific type of the target subject, and when the electronic device receives the image attribute identification request, the target subject in the image to be identified can be obtained, for example, there are multiple people in a group photo, and the subject to be identified can be extracted; there are multiple plants or animals in a landscape photo, and the target image to be identified can be extracted. In the process of image attribute identification, non-identification objects are excluded and the target subject is retained.

[0124] 304、obtaining the target image in the image to be identified according to the target subject.

[0125] In some embodiments, after the target subject is obtained, the image in which the target subject is located can be cropped to obtain the target image, which can avoid the interference of non-identification subjects on the identification of the target subject in the process of image attribute identification, so that the identification speed is faster and the identification result is more accurate in the process of image attribute identification.

[0126] 305、calling a pre-trained image attribute identification model.

[0127] The image attribute identification model is obtained by training the network model provided in the embodiment using the training method. The training process of the specific network model can be referred to the related description of the above-mentioned embodiments, which will not be repeated here.

[0128] 306、inputting the target image into the pre-trained image attribute identification model to obtain an image attribute identification result.

[0129] After the target image is input into the image attribute identification model, the image attribute identification is performed to obtain the identification result of multiple attributes in the image and the correlation between the attributes.

[0130] For example, in the identified human body images, the image attribute recognition model can identify the correlation between human clothing, such as the correlation between shorts and short-sleeved shirts is 100%, the correlation between jeans and sneakers is 80%, the correlation between hats and glasses is 50%, and so on. In this way, the correlation between various wearing items and clothing is obtained, and users can better refer to how to match clothing.

[0131] From the above, it can be seen that the image attribute recognition method provided in the embodiment of the present application receives an image attribute recognition request, obtains the image to be recognized according to the image attribute recognition request, and then calls a pre-trained image attribute recognition model, inputs the image to be recognized into the pre-trained image attribute recognition model, and recognizes the image attributes of the image to be recognized to obtain an image attribute recognition result, thereby obtaining the correlation between the various attributes of the image.

[0132] See also Figure 4 , Figure 4 4 is a schematic diagram of the structure of the network model training device provided in an embodiment of the present application. The network model training device 400 may include: a first acquisition module 410, a construction module 420, a first recognition module 430, a training module 440 and a determination module 450.

[0133] A first acquisition module 410 is configured to acquire an image sample set, wherein the image sample set includes a plurality of initial values ​​of image attributes;

[0134] A construction module 420 is used to construct a basic model and a target loss function corresponding to the basic model, wherein the basic model includes a convolutional neural network model and a recurrent neural network model;

[0135] A first recognition module 430 is configured to input the image sample set into the basic model to perform image attribute recognition to obtain a first training result obtained according to the recurrent neural network model and a second training result obtained according to the convolutional neural network model;

[0136] A training module 440 is configured to jointly train the convolutional neural network model and the recurrent neural network model according to the first training result, the second training result, the initial value of the image attribute, and the target loss function until the basic model converges;

[0137] The determination module 450 is configured to use the converged basic model as a recognition model for recognizing image attributes.

[0138] In some embodiments, the constructing module 420 includes a setting submodule 421, a first connecting submodule 422, and a second connecting submodule 423, wherein the setting submodule 421 is configured to set a convolutional layer, a pooling layer, a first fully connected layer, and a second fully connected layer; the first connecting submodule 422 is configured to connect the convolutional layer, the pooling layer, and the second fully connected layer to obtain the convolutional neural network model; and the second connecting submodule 423 is configured to connect the recurrent neural network with the convolutional layer and connect the recurrent neural network with the first fully connected layer to obtain the recurrent neural network model.

[0139] The constructing module 420 is specifically configured to construct a first loss function corresponding to the convolutional neural network model, construct a second loss function corresponding to the recurrent neural network model, and obtain a target loss function corresponding to the base model according to the first loss function and the second loss function. For example, the second loss function can be multiplied by a loss coefficient to obtain a target second loss function, and the target second loss function can be added to the first loss function to obtain the target loss function.

[0140] In some embodiments, the training module 440 is specifically configured to input the first training result, the second training result, and the image attribute initial value into the target loss function to obtain a target loss value, and adjust parameters of the base model according to the target loss value.

[0141] In some embodiments, the determining module 450 is specifically configured to input the image sample set into the convolutional layer to obtain the first feature value, input the first feature value into the recurrent neural network to obtain the second feature value, and input the second feature value into the first fully connected layer to obtain the first training result.

[0142] As known from the above, the first acquiring module 410 acquires an image sample set, the image sample set including a plurality of image attribute initial values; the constructing module 420 constructs a base model and a target loss function corresponding to the base model, the base model including a convolutional neural network model and a recurrent neural network model; the first identifying module 430 inputs the image sample set into the base model to perform image attribute identification, to obtain a first training result obtained according to the recurrent neural network model and a second training result obtained according to the convolutional neural network model; the training module 440 jointly trains the convolutional neural network model and the recurrent neural network model according to the first training result, the second training result, the image attribute initial value, and the target loss function, until the base model converges; and the determining module 450 takes the converged base model as an identification model for identifying image attributes. The trained base network model can improve the accuracy of image attribute identification and identify the relevance between various image attributes.

[0143] It should be noted that the network model training device provided in the embodiment of the present application and the network model training method in the above embodiment belong to the same concept. Any method provided in the network model training method embodiment can be run on the network model training device. The specific implementation process is detailed in the network model training method embodiment, which will not be repeated here.

[0144] See also Figure 5 The image attribute recognition device 500 may include a receiving module 510 , a second acquiring module 520 , a calling module 530 , and a second recognition module 540 .

[0145] Receiving module 510, for receiving an image attribute recognition request;

[0146] A second acquisition module 520 is configured to acquire an image to be identified according to the image attribute identification request;

[0147] A calling module 530 is used to call a pre-trained image attribute recognition model;

[0148] The second recognition module 540 is configured to input the image to be recognized into a pre-trained image attribute recognition model, recognize the image attributes of the image to be recognized, and obtain an image attribute recognition result.

[0149] In some embodiments, the second acquisition module 520 is specifically configured to identify a target subject in the image to be identified according to an image attribute recognition request, and acquire a target image in the image to be identified according to the target subject.

[0150] As can be seen from the above, the image attribute recognition device 500 provided in the embodiment of the present application receives an image attribute recognition request via a receiving module 510; a second acquisition module 520 obtains the image to be recognized based on the image attribute recognition request; a calling module 530 calls a pre-trained image attribute recognition model; and a second recognition module 540 inputs the image to be recognized into the pre-trained image attribute recognition model to recognize the image attributes of the image to be recognized, thereby obtaining an image attribute recognition result. The image attribute recognition device trained using the above-mentioned network model training method can accurately identify the various attributes in an image and the correlations between the attributes, thereby improving the accuracy of image attribute recognition.

[0151] It should be noted that the image attribute recognition device provided in the embodiment of the present application and the image attribute recognition method in the above embodiment belong to the same concept. Any method provided in the image attribute recognition method embodiment can be run on the image attribute recognition device. The specific implementation process is detailed in the image processing method embodiment and will not be repeated here.

[0152] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the stored computer program is executed on a computer, the computer executes a network model training method or an image processing method as provided in an embodiment of the present application.

[0153] The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0154] An embodiment of the present application also provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is used to execute the network model training method or image attribute recognition method provided in the embodiment of the present application by calling the computer program stored in the memory.

[0155] For example, the electronic device may be a mobile terminal such as a tablet computer or a smart phone. Figure 6 , Figure 6 This is a schematic diagram of the first structure of the electronic device provided in an embodiment of the present application.

[0156] The electronic device 600 may include components such as a memory 601 and a processor 602. Those skilled in the art will appreciate that Figure 7 The electronic device structure shown in the figure does not constitute a limitation to the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0157] The memory 601 can be used to store software programs and modules. The processor 602 executes various functional applications and data processing by running the computer programs and modules stored in the memory 601. The memory 601 can mainly include a program storage area and a data storage area. The program storage area can store an operating system and computer programs required for at least one function (such as a sound playback function, an image playback function, etc.); the data storage area can store data created based on the use of the electronic device.

[0158] The processor 602 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. By running or executing applications stored in the memory 601 and calling data stored in the memory 601, it performs various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole.

[0159] In addition, the memory 601 may include a high-speed random access memory and a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory 601 may also include a memory controller to provide the processor 602 with access to the memory 601.

[0160] In this embodiment, the processor 602 in the electronic device loads the executable code corresponding to one or more application processes into the memory 601 according to the following instructions, and the processor 602 runs the application stored in the memory 601, thereby implementing the process:

[0161] Acquire an image sample set, wherein the image sample set includes a plurality of image attribute initial values;

[0162] Constructing a basic model and a target loss function corresponding to the basic model, wherein the basic model includes a convolutional neural network model and a recurrent neural network model;

[0163] Inputting the image sample set into the basic model to perform image attribute recognition to obtain a first training result obtained according to the recurrent neural network model and a second training result obtained according to the convolutional neural network model;

[0164] Jointly training the convolutional neural network model and the recurrent neural network model according to the first training result, the second training result, the initial value of the image attribute, and the target loss function until the basic model converges;

[0165] The converged basic model is used as a recognition model for recognizing image attributes.

[0166] In some implementations, when the processor 602 executes the construction of the target loss function corresponding to the base model, it may execute:

[0167] Constructing a first loss function corresponding to the convolutional neural network model;

[0168] Constructing a second loss function corresponding to the recurrent neural network model;

[0169] The target loss function corresponding to the basic model is obtained according to the first loss function and the second loss function.

[0170] Specifically, when the processor 602 obtains the target loss function corresponding to the basic model according to the first loss function and the second loss function, it can execute:

[0171] Multiplying the second loss function by the loss coefficient to obtain a target second loss function;

[0172] The target loss function is obtained by adding the target second loss function to the first loss function.

[0173] In some embodiments, the processor 602 performs joint training of the convolutional neural network model and the recurrent neural network model according to the first training result, the second training result, the initial value of the image attribute, and the target loss function until the base model converges, and may perform:

[0174] Inputting the first training result, the second training result, and the initial value of the image attribute into the target loss function to obtain a target loss value;

[0175] The parameters of the basic model are adjusted according to the target loss value.

[0176] In some implementations, when the processor 602 executes the construction of the base model, it may execute:

[0177] Set up the convolutional layer, pooling layer, first fully connected layer and second fully connected layer;

[0178] Connecting the convolutional layer, the pooling layer, and the second fully connected layer to obtain the convolutional neural network model;

[0179] The recurrent neural network is connected to the convolutional layer, and the recurrent neural network is connected to the first fully connected layer to obtain the recurrent neural network model.

[0180] In some embodiments, when the processor 602 inputs the image sample set into the basic model to perform image attribute recognition to obtain a first training result obtained according to the recurrent neural network model, the processor 602 may execute:

[0181] Inputting the image sample set into the convolutional layer to obtain the first eigenvalue;

[0182] Inputting the first eigenvalue into the recurrent neural network to obtain the second eigenvalue;

[0183] The second feature value is input into the first fully connected layer to obtain the first training result.

[0184] In this embodiment, the processor 602 in the electronic device loads the executable code corresponding to one or more application processes into the memory 601 according to the following instructions, and the processor 602 runs the application stored in the memory 601, thereby implementing the process:

[0185] receiving an image attribute recognition request;

[0186] Acquire an image to be identified according to the image attribute identification request;

[0187] Call the pre-trained image attribute recognition model;

[0188] The image to be identified is input into a pre-trained image attribute recognition model, and the image attributes of the image to be identified are identified to obtain an image attribute recognition result.

[0189] In some embodiments, when the processor 602 obtains the image to be identified according to the image attribute identification request, it may execute:

[0190] Identify the target subject in the image to be identified according to the image attribute recognition request;

[0191] A target image in the image to be identified is acquired according to the target subject.

[0192] Please refer to Figure 7 , Figure 7 A second structural diagram of an electronic device provided in an embodiment of the present application, Figure 6 The difference between the electronic device shown is that the electronic device further includes: a camera component, a radio frequency circuit 604, an audio circuit 605 and a power supply 606. Among them, the display 603, the radio frequency circuit 604, the audio circuit 605 and the power supply 606 are electrically connected to the processor 602 respectively.

[0193] The display 603 can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces. These graphical user interfaces can be composed of graphics, text, icons, videos, or any combination thereof. The display 603 may include a display panel. In some embodiments, the display panel can be configured in the form of a liquid crystal display (LCD) or an organic light-emitting diode (OLED).

[0194] The radio frequency circuit 604 can be used to transmit and receive radio frequency signals, so as to establish wireless communication with a network device or other electronic devices through wireless communication, and to transmit and receive signals with the network device or other electronic devices.

[0195] The audio circuit 605 may be configured to provide an audio interface between a user and the electronic device through a speaker and a microphone.

[0196] The power supply 606 can be used to supply power to various components of the electronic device 600. In some embodiments, the power supply 606 can be logically connected to the processor 602 through a power management system, thereby managing charging, discharging, and power consumption through the power management system.

[0197] although Figure 7 Not shown, the electronic device 600 may also include a camera component, a Bluetooth module, etc. The camera component may include an image processing circuit. The image processing circuit may be implemented using hardware and / or software components and may include various processing units that define an image signal processing (Image Signal Processing) pipeline. The image processing circuit may include at least: multiple cameras, an image signal processor (ISP processor), a control logic, an image memory, and a display, etc. Each camera may include at least one or more lenses and an image sensor. The image sensor may include a color filter array (such as a Bayer filter). The image sensor may obtain light intensity and wavelength information captured by each imaging pixel of the image sensor, and provide a set of raw image data that can be processed by an image signal processor.

[0198] In the above embodiments, the description of each embodiment has its own focus. For the parts that are not described in detail in a certain embodiment, please refer to the detailed description of the training method / image processing method of the network model above, which will not be repeated here.

[0199] The network model training method / image processing method device provided in the embodiment of the present application belongs to the same concept as the network model training method / image processing method in the above embodiment. Any method provided in the network model training method / image processing method embodiment can be run on the network model training method / image processing method device. The specific implementation process is detailed in the network model training method / image processing method embodiment, which will not be repeated here.

[0200] It should be noted that, for the network model training method / image processing method described in the embodiment of the present application, ordinary technicians in this field can understand that all or part of the process of implementing the network model training method / image processing method described in the embodiment of the present application can be completed by controlling the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, such as a memory, and executed by at least one processor. During the execution process, it may include the process of the embodiment of the network model training method / image processing method. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), etc.

[0201] For the training method / image processing method device of the network model in the embodiment of the present application, its various functional modules can be integrated into a processing chip, or each module can exist physically separately, or two or more modules can be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software functional modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium, such as a read-only memory, a disk or an optical disk.

[0202] The above is a detailed introduction to a network model training method, image processing method, device, storage medium and electronic device provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, based on the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A network model training method, characterized in that: The method comprises: Acquire an image sample set, wherein the image sample set includes a plurality of image attribute initial values; Constructing a basic model and a target loss function corresponding to the basic model, wherein the basic model includes a convolutional neural network model and a recurrent neural network model; Inputting the image sample set into the basic model to perform image attribute recognition to obtain a first training result obtained according to the recurrent neural network model and a second training result obtained according to the convolutional neural network model; Jointly training the convolutional neural network model and the recurrent neural network model according to the first training result, the second training result, the initial value of the image attribute, and the target loss function until the basic model converges; The converged basic model is used as a recognition model for recognizing image attributes.

2. The network model training method according to claim 1, characterized in that The constructing of the target loss function corresponding to the basic model includes: Constructing a first loss function corresponding to the convolutional neural network model; Constructing a second loss function corresponding to the recurrent neural network model; The target loss function corresponding to the basic model is obtained according to the first loss function and the second loss function.

3. The network model training method according to claim 2, characterized in that The obtaining of the target loss function corresponding to the basic model according to the first loss function and the second loss function includes: Multiplying the second loss function by the loss coefficient to obtain a target second loss function; The target loss function is obtained by adding the target second loss function to the first loss function.

4. The network model training method according to claim 3, characterized in that The jointly training the convolutional neural network model and the recurrent neural network model according to the first training result, the second training result, the initial value of the image attribute, and the target loss function until the basic model converges includes: Inputting the first training result, the second training result, and the initial value of the image attribute into the target loss function to obtain a target loss value; The parameters of the basic model are adjusted according to the target loss value.

5. The network model training method according to claim 1, characterized in that The construction of the basic model includes: Set up the convolutional layer, pooling layer, first fully connected layer and second fully connected layer; Connecting the convolutional layer, the pooling layer, and the second fully connected layer to obtain the convolutional neural network model; The recurrent neural network model is connected to the convolutional layer, and the recurrent neural network model is connected to the first fully connected layer to obtain the recurrent neural network model.

6. The network model training method according to claim 5, characterized in that Inputting the image sample set into the basic model to perform image attribute recognition to obtain a first training result obtained according to the recurrent neural network model includes: Inputting the image sample set into the convolutional layer to obtain the first eigenvalue; Inputting the first eigenvalue into the recurrent neural network model to obtain the second eigenvalue; The second feature value is input into the first fully connected layer to obtain the first training result.

7. A method for image attribute recognition, characterized in that: The method comprises: receiving an image attribute recognition request; Acquire an image to be identified according to the image attribute identification request; Call the pre-trained image attribute recognition model; Inputting the image to be identified into the pre-trained image attribute recognition model to identify the image attributes of the image to be identified to obtain an image attribute recognition result; Wherein, the image attribute recognition model is an image attribute recognition model trained using the network model training method according to any one of claims 1 to 6.

8. The image attribute recognition method according to claim 7, characterized in that: The acquiring of the image to be identified according to the image attribute identification request includes: Identify the target subject in the image to be identified according to the image attribute recognition request; A target image in the image to be identified is acquired according to the target subject.

9. A network model training device, characterized in that: include: A first acquisition module is used to acquire an image sample set, wherein the image sample set includes a plurality of initial values ​​of image attributes; A construction module is used to construct a basic model and a target loss function corresponding to the basic model, wherein the basic model includes a convolutional neural network model and a recurrent neural network model; a first recognition module, configured to input the image sample set into the basic model to perform image attribute recognition, so as to obtain a first training result obtained according to the recurrent neural network model and a second training result obtained according to the convolutional neural network model; a training module, configured to jointly train the convolutional neural network model and the recurrent neural network model according to the first training result, the second training result, the initial value of the image attribute, and the target loss function until the basic model converges; A determination module is used to use the converged basic model as a recognition model for recognizing image attributes.

10. The training device according to claim 9, characterized in that The building blocks include: Set up submodules for setting up convolutional layers, pooling layers, first fully connected layers, and second fully connected layers; A first connection submodule, configured to connect the convolutional layer, the pooling layer, and the second fully connected layer to obtain the convolutional neural network model; The second connection submodule is used to connect the recurrent neural network model with the convolutional layer, and to connect the recurrent neural network model with the first fully connected layer to obtain the recurrent neural network model.

11. A device for identifying image attributes, characterized in that: include: A receiving module, configured to receive an image attribute recognition request; A second acquisition module is used to acquire the image to be identified according to the image attribute recognition request; The calling module is used to call the pre-trained image attribute recognition model; a second recognition module, configured to input the image to be recognized into the pre-trained image attribute recognition model, recognize the image attributes of the image to be recognized, and obtain an image attribute recognition result; Wherein, the image attribute recognition model is an image attribute recognition model obtained by using the network model training method according to any one of claims 1 to 6.

12. A storage medium, characterized in that: The storage medium stores a computer program, which, when executed on a computer, enables the computer to execute the network model training method according to any one of claims 1 to 6 or the image attribute recognition method according to claims 7 and 8.

13. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores a computer program, and the processor calls the computer program stored in the memory to execute: Acquire an image sample set, wherein the image sample set includes a plurality of image attribute initial values; Constructing a basic model and a target loss function corresponding to the basic model, wherein the basic model includes a convolutional neural network model and a recurrent neural network model; Inputting the image sample set into the basic model to perform image attribute recognition to obtain a first training result obtained according to the recurrent neural network model and a second training result obtained according to the convolutional neural network model; Jointly training the convolutional neural network model and the recurrent neural network model according to the first training result, the second training result, the initial value of the image attribute, and the target loss function until the basic model converges; The converged basic model is used as a recognition model for recognizing image attributes.

14. The electronic device according to claim 13, wherein: The processor is configured to execute: Constructing a first loss function corresponding to the convolutional neural network model; Constructing a second loss function corresponding to the recurrent neural network model; The target loss function corresponding to the basic model is obtained according to the first loss function and the second loss function.

15. The electronic device according to claim 14, characterized in that The processor is configured to execute: Multiplying the second loss function by the loss coefficient to obtain a target second loss function; The target loss function is obtained by adding the target second loss function to the first loss function.

16. The electronic device according to claim 15, characterized in that The processor is configured to execute: Inputting the first training result, the second training result, and the initial value of the image attribute into the target loss function to obtain a target loss value; The parameters of the basic model are adjusted according to the target loss value.

17. The electronic device according to claim 13, wherein: The processor is configured to execute: Set up the convolutional layer, pooling layer, first fully connected layer and second fully connected layer; Connecting the convolutional layer, the pooling layer, and the second fully connected layer to obtain the convolutional neural network model; The recurrent neural network model is connected to the convolutional layer, and the recurrent neural network model is connected to the first fully connected layer to obtain the recurrent neural network model. The recurrent neural network model and the convolutional neural network model are combined to form the basic model.

18. The electronic device according to claim 17, wherein: The processor is configured to execute: Inputting the image sample set into the convolutional layer to obtain the first eigenvalue; Inputting the first eigenvalue into the recurrent neural network model to obtain the second eigenvalue; The second feature value is input into the first fully connected layer to obtain the first training result.

19. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores a computer program, and the processor calls the computer program stored in the memory to execute: receiving an image attribute recognition request; Acquire an image to be identified according to the image attribute identification request; Call the pre-trained image attribute recognition model; Inputting the image to be identified into the pre-trained image attribute recognition model to identify the image attributes of the image to be identified to obtain an image attribute recognition result; Wherein, the image attribute recognition model is an image attribute recognition model trained using the network model training method according to any one of claims 1 to 6.

20. The electronic device according to claim 19, wherein The processor is configured to execute: Identify the target subject in the image to be identified according to the image attribute recognition request; A target image in the image to be identified is acquired according to the target subject.

Images