A method and device for constructing a training data set

Abstract

The present invention discloses a method and device for constructing a training data set, and relates to the field of computer technology. The method comprises: randomly selecting two images to be annotated from the data set to be annotated as the first image to be annotated and the second image to be annotated, and determining the first basic aesthetic label of the first image to be annotated and the second basic aesthetic label of the second image to be annotated; using the first image to be annotated and the second image to be annotated as input data, inputting a pre-trained image aesthetic comparison model, and obtaining a first evaluation factor and a second evaluation factor; updating the first basic aesthetic label and the second basic aesthetic label according to the first evaluation factor and the second evaluation factor; repeating the above steps until the number of repetitions reaches a threshold, and using the finally obtained basic aesthetic label as the final aesthetic label of the image to be annotated, so as to obtain a training data set. The method and device reduce the complexity and cost of annotation, and improve scalability.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method and device for constructing a training data set. Background Art

[0002] Image aesthetic evaluation measures the visual appeal of an image in the human eye. Currently, image aesthetic evaluation algorithms mainly use deep neural network deep learning to directly regress image aesthetic scores on large data sets.

[0003] In the process of implementing the present invention, the inventors found that there are at least the following problems in the prior art:

[0004] In the deep learning method, the deep neural network needs to be trained on the image aesthetics dataset, and the dataset annotation method requires a lot of manpower cost. Moreover, for images of different categories, a training set needs to be annotated for each category of images, which will be a very expensive task. Summary of the invention

[0005] In view of this, an embodiment of the present invention provides a method and device for constructing a training data set, which can reduce the complexity and cost of annotation and improve scalability.

[0006] To achieve the above object, according to one aspect of an embodiment of the present invention, a method for constructing a training data set is provided, comprising:

[0007] Randomly select two images to be labeled from the to-be-labeled data set as a first to-be-labeled image and a second to-be-labeled image, and determine a first basic aesthetic label for the first to-be-labeled image and a second basic aesthetic label for the second to-be-labeled image;

[0008] The first image to be annotated and the second image to be annotated are used as input data and input into a pre-trained image aesthetic comparison model to obtain a first evaluation factor of the first image to be annotated and a second evaluation factor of the second image to be annotated;

[0009] updating the first basic aesthetic label and the second basic aesthetic label according to the first evaluation factor and the second evaluation factor;

[0010] Repeat the above steps until the number of repetitions reaches a threshold, and use the final basic aesthetic label as the final aesthetic label of the image to be labeled to obtain a training data set.

[0011] Optionally, determining a first basic aesthetic label of the first image to be labeled and a second basic aesthetic label of the second image to be labeled includes:

[0012] The first image to be labeled and the second image to be labeled are respectively used as input data and input into a pre-trained basic aesthetic evaluation model to obtain a first basic aesthetic label of the first image to be labeled and a second basic aesthetic label of the second image to be labeled.

[0013] Optionally, the image aesthetic comparison model is trained according to the following process:

[0014] Obtain multiple sets of image pairs for various purposes, and manually annotate each set of image pairs to generate an aesthetic comparison training set;

[0015] The aesthetic comparison training set is trained to obtain an image aesthetic comparison model.

[0016] Optionally, training the aesthetic comparison training set to obtain the image aesthetic comparison model includes: setting multiple training methods and multiple model complexities; and training the aesthetic comparison training set based on the multiple training methods and multiple model complexities to obtain multiple image aesthetic comparison models.

[0017] Optionally, the first image to be annotated and the second image to be annotated are used as input data and input into a pre-trained image aesthetic comparison model to obtain a first evaluation factor of the first image to be annotated and a second evaluation factor of the second image to be annotated, including:

[0018] Taking the first image to be labeled and the second image to be labeled as input data, inputting them into one of the multiple image aesthetic comparison models, and obtaining a first evaluation factor of the first image to be labeled and a second evaluation factor of the second image to be labeled;

[0019] Updating the first basic aesthetic label and the second basic aesthetic label according to the first evaluation factor and the second evaluation factor includes:

[0020] updating the first basic aesthetic label and the second basic aesthetic label according to the first evaluation factor and the second evaluation factor, and using the updated first basic aesthetic label and the updated second basic aesthetic label as a new first basic aesthetic label and a new second basic aesthetic label, respectively;

[0021] The first image to be annotated and the second image to be annotated are used as input data and input into other image aesthetic comparison models to obtain a new first evaluation factor and a new second evaluation factor;

[0022] Based on the new first evaluation factor and the new second evaluation factor, updating the new first basic aesthetic label and the new second basic aesthetic label;

[0023] The above steps are repeated until the multiple image aesthetic comparison models are traversed.

[0024] Optionally, the first basic aesthetic label and the second basic aesthetic label are updated according to the following formula:

[0025]

[0026] Among them, Ra represents the first basic aesthetic label, Rb represents the second basic aesthetic label, Sa represents the first evaluation factor, Sb represents the second evaluation factor, k and β are preset parameters, Ra * Represents the updated first base aesthetic label, Rb * Represents the updated second base aesthetic label.

[0027] Optionally, using the finally obtained basic aesthetic label as the final aesthetic label of the image to be annotated includes: normalizing the finally obtained basic aesthetic label, and using the normalized basic aesthetic label as the final aesthetic label of the image to be annotated.

[0028] To achieve the above object, according to another aspect of an embodiment of the present invention, a device for constructing a training data set is provided, comprising:

[0029] A basic label acquisition module is used to randomly select two images to be labeled from the to-be-labeled data set as a first to-be-labeled image and a second to-be-labeled image, and determine a first basic aesthetic label for the first to-be-labeled image and a second basic aesthetic label for the second to-be-labeled image;

[0030] An evaluation factor acquisition module, used to take the first image to be annotated and the second image to be annotated as input data, input into a pre-trained image aesthetic comparison model, and obtain a first evaluation factor of the first image to be annotated and a second evaluation factor of the second image to be annotated;

[0031] A label updating module is used to update the first basic aesthetic label and the second basic aesthetic label according to the first evaluation factor and the second evaluation factor, repeat the above steps until the number of repetitions reaches a threshold, and use the finally obtained basic aesthetic label as the final aesthetic label of the image to be annotated to obtain a training data set.

[0032] Optionally, the basic label acquisition module takes the first image to be labeled and the second image to be labeled as input data respectively, inputs them into a pre-trained basic aesthetic evaluation model, and obtains a first basic aesthetic label for the first image to be labeled and a second basic aesthetic label for the second image to be labeled.

[0033] Optionally, the image aesthetic comparison model is trained according to the following process:

[0034] Obtain multiple sets of image pairs for various purposes, and manually annotate each set of image pairs to generate an aesthetic comparison training set;

[0035] The aesthetic comparison training set is trained to obtain an image aesthetic comparison model.

[0036] Optionally, training the aesthetic comparison training set to obtain an image aesthetic comparison model includes:

[0037] A plurality of training methods and a plurality of model complexities are set; based on the plurality of training methods and the plurality of model complexities, the aesthetic comparison training set is trained to obtain a plurality of image aesthetic comparison models.

[0038] Optionally, the evaluation factor acquisition module is further used to:

[0039] Taking the first image to be labeled and the second image to be labeled as input data, inputting them into one of the multiple image aesthetic comparison models, and obtaining a first evaluation factor of the first image to be labeled and a second evaluation factor of the second image to be labeled;

[0040] The label update module is also used for:

[0041] updating the first basic aesthetic label and the second basic aesthetic label according to the first evaluation factor and the second evaluation factor, and using the updated first basic aesthetic label and the updated second basic aesthetic label as a new first basic aesthetic label and a new second basic aesthetic label, respectively;

[0042] The first image to be annotated and the second image to be annotated are used as input data and input into other image aesthetic comparison models to obtain a new first evaluation factor and a new second evaluation factor;

[0043] Based on the new first evaluation factor and the new second evaluation factor, updating the new first basic aesthetic label and the new second basic aesthetic label;

[0044] The above steps are repeated until the multiple image aesthetic comparison models are traversed.

[0045] Optionally, the label updating module is further used to update the first basic aesthetic label and the second basic aesthetic label according to the following formula:

[0046]

[0047] Among them, Ra represents the first basic aesthetic label, Rb represents the second basic aesthetic label, Sa represents the first evaluation factor, Sb represents the second evaluation factor, k and β are preset parameters, Ra * Represents the updated first base aesthetic label, Rb * Represents the updated second base aesthetic label.

[0048] Optionally, the label updating module is further used to: normalize the finally obtained basic aesthetic label, and use the normalized basic aesthetic label as the final aesthetic label of the image to be annotated.

[0049] To achieve the above-mentioned purpose, according to another aspect of an embodiment of the present invention, there is provided an electronic device, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method for constructing a training data set according to an embodiment of the present invention.

[0050] To achieve the above objective, according to another aspect of an embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored, and when the program is executed by a processor, the method for constructing a training data set according to an embodiment of the present invention is implemented.

[0051] An embodiment of the above invention has the following advantages or beneficial effects: by randomly selecting two images to be labeled from a data set to be labeled as a first image to be labeled and a second image to be labeled, and determining a first basic aesthetic label of the first image to be labeled and a second basic aesthetic label of the second image to be labeled; using the first image to be labeled and the second image to be labeled as input data, inputting a pre-trained image aesthetic comparison model, and obtaining a first evaluation factor of the first image to be labeled and a second evaluation factor of the second image to be labeled; updating the first basic aesthetic label and the second basic aesthetic label according to the first evaluation factor and the second evaluation factor, repeating the above steps until the number of repetitions reaches a threshold, and using the finally obtained basic aesthetic label as the final aesthetic label of the image to be labeled to obtain a technical means of obtaining a training data set, thereby reducing the complexity and cost of labeling when constructing a training data set and improving scalability.

[0052] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0054] Figure 1 is a schematic diagram of the main process of the method for constructing a training data set according to an embodiment of the present invention;

[0055] Figure 2 is a diagram of a process of a method for constructing a training data set according to an embodiment of the present invention;

[0056] Figure 3 is a schematic diagram of a sub-process of a method for constructing a training data set according to an embodiment of the present invention;

[0057] Figure 4 is a flowchart of a method for constructing a training data set according to another embodiment of the present invention;

[0058] Figure 5 is a schematic diagram of main modules of a device for constructing a training data set according to an embodiment of the present invention;

[0059] Figure 6 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0060] Figure 7 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0061] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0062] Figure 1 FIG. 1 is a schematic diagram of the main process of the method for constructing a training data set according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0063] Step S101: randomly selecting two images to be labeled from the to-be-labeled data set as a first to-be-labeled image and a second to-be-labeled image, and determining a first basic aesthetic label for the first to-be-labeled image and a second basic aesthetic label for the second to-be-labeled image.

[0064] In this step, the first basic aesthetic label and the second basic aesthetic label can be obtained through the following process:

[0065] The first image to be annotated and the second image to be annotated are respectively used as input data and input into a pre-trained basic aesthetic evaluation model to obtain a first basic aesthetic label for the first image to be annotated and a second basic aesthetic label for the second image to be annotated. The basic aesthetic evaluation model can be a general model trained on a public aesthetic dataset, such as a general model trained on an AVA aesthetic dataset (The Aesthetic Visual Analysis dataset, a public aesthetic dataset). Due to the diverse image types and fine annotation results of the AVA aesthetic dataset, the image aesthetic evaluation given by the model trained thereon has good generalization performance. In other optional embodiments, the basic aesthetic evaluation model can also use other public aesthetic datasets, and the present invention is not limited here. When the basic aesthetic evaluation model is obtained by training the AVA aesthetic dataset, the first basic aesthetic label and the second basic aesthetic label are aesthetic scores within 1-10 respectively.

[0066] The first image to be annotated and the second image to be annotated may be from the same category or from different categories. The category to which the image belongs may be determined based on the category to which the object described in the image belongs. For example, the image category may be divided based on the result of the classification of commodity categories. When classifying the image category, the granularity of the division may be set according to the application requirements.

[0067] Step S102: taking the first image to be annotated and the second image to be annotated as input data and inputting them into a pre-trained image aesthetic comparison model, to obtain a first evaluation factor of the first image to be annotated and a second evaluation factor of the second image to be annotated.

[0068] In this step, the image aesthetic comparison model can be trained according to the following process:

[0069] Obtain multiple sets of image pairs for various purposes, and manually annotate each set of image pairs to generate an aesthetic comparison training set;

[0070] The aesthetic comparison training set is trained to obtain an image aesthetic comparison model.

[0071] Before training the image aesthetic comparison model, it is necessary to annotate the data set used to train the image aesthetic comparison model, namely the aesthetic comparison training set. The annotation process is as follows: for each category of images, obtain a batch of image pairs of the category; then manually select the more beautiful image in the image pair and annotate it as the first image to obtain the first evaluation factor, such as 1; annotate the other image in the image pair as the second image to obtain the second evaluation factor, such as 0; finally, use the annotated image pairs of all categories for training (during training, two images are used as input data for training, and the two images can be from the same category or not), thereby obtaining the image aesthetic comparison model.

[0072] It is worth noting that the image pairs labeled in all categories in this step are the only training sets that need to be labeled in the method for constructing a training data set in the embodiment of the present invention. Moreover, unlike the prior art, the labeling only needs to select the more beautiful one in the image pair, and does not need to label the aesthetic evaluation score of 1-10 points. Therefore, the method for constructing a training data set in the embodiment of the present invention reduces the cost of constructing a training set and is easier to expand to different image categories by converting the original method of 1-10 points classification labeling to a simple two-choice method of labeling the more beautiful and attractive image pairs.

[0073] In the above embodiment, for the convenience of labeling, image pairs from the same category are used as objects for labeling. In other optional embodiments, image pairs from the same category may not be used as objects for labeling, that is, the two images of the image pair may be from different categories.

[0074] Step S103: updating the first basic aesthetic label and the second basic aesthetic label according to the first evaluation factor and the second evaluation factor.

[0075] As a specific example, the first basic aesthetic label and the second basic aesthetic label may be adjusted using the ELO algorithm (Elo rating system). Specifically, the first basic aesthetic label and the second basic aesthetic label may be updated according to the following formula (1):

[0076]

[0077] Among them, Ra represents the first basic aesthetic label, Rb represents the second basic aesthetic label, Sa represents the first evaluation factor, Sb represents the second evaluation factor, k and β are preset parameters, Ra * Represents the updated first base aesthetic label, Rb * Represents the updated second base aesthetic label.

[0078] The specific derivation principle of this formula is as follows:

[0079] Sa: the first evaluation factor of the first image to be annotated under the image aesthetic comparison model, that is, the actual score value (0 or 1);

[0080] Sb: The second evaluation factor of the second image to be annotated under the image aesthetic comparison model, Sb = 1-Sa;

[0081] Ea: expected comparison score of the first image to be annotated, Ea = 1 / (1+10^[(Rb-Ra) / 400]);

[0082] Eb: the expected comparison score of the second image to be annotated, Eb = 1-Ea;

[0083] The first basic aesthetic label Ra of the first image to be labeled after adjustment * =Ra+K(Sa-Ea);

[0084] The second basic aesthetic label Rb of the second image to be labeled after adjustment * =Rb+K(Sb-Eb);

[0085] Substituting Ea and Eb into Ra * and Rb * In the calculation formula of , we can get formula (1).

[0086] As a specific example, β can be set to 400. In other optional embodiments, β can also be set to other values. K is a proportional coefficient, which can be adjusted according to the range of the final aesthetic label (i.e., aesthetic score) set. The size of K will affect the speed at which the image aesthetic score changes. If the aesthetic score difference between the two pictures needs to be larger, K can be set larger, otherwise, K can be set smaller.

[0087] Step S104: Repeat the above steps S101-S103 until the number of repetitions reaches a threshold, and use the finally obtained basic aesthetic label as the final aesthetic label of the image to be annotated to obtain a training data set.

[0088] In this embodiment, the number of loop executions may be preset, and the number may be much greater than the number of images to be annotated in the data set to be annotated.

[0089] In an embodiment of the present invention, if there are M images to be annotated in the data set to be annotated (M is an integer greater than or equal to 2). When randomly selecting the first image to be annotated and the second image to be annotated, the selected image may be any image to be annotated in the data set to be annotated. For the image to be annotated selected for the first time, its basic aesthetic label is obtained through the basic aesthetic evaluation model. For the image to be annotated that is not the first selected, its basic aesthetic label is obtained by updating by cyclically executing steps S101-S103. By cyclically executing steps S101-S103, the embodiment of the present invention can make the image annotation more accurate.

[0090] The method for constructing a training data set in an embodiment of the present invention randomly selects two images to be labeled from the data set to be labeled as a first image to be labeled and a second image to be labeled, and determines a first basic aesthetic label of the first image to be labeled and a second basic aesthetic label of the second image to be labeled; uses the first image to be labeled and the second image to be labeled as input data, inputs a pre-trained image aesthetic comparison model, and obtains a first evaluation factor of the first image to be labeled and a second evaluation factor of the second image to be labeled; updates the first basic aesthetic label and the second basic aesthetic label according to the first evaluation factor and the second evaluation factor, repeats the above steps until the number of repetitions reaches a threshold, and uses the finally obtained basic aesthetic label as the final aesthetic label of the image to be labeled to obtain a technical means of the training data set, thereby reducing the complexity and cost of labeling when constructing the training data set and improving scalability.

[0091] In an optional embodiment, when training the aesthetic comparison training set, different training methods and different model complexities may be set to obtain multiple image aesthetic comparison models.

[0092] When multiple image aesthetic comparison models are trained, such as Figure 3 As shown, step S102 and step S103 may include the following sub-steps:

[0093] Step S301: inputting the first image to be annotated and the second image to be annotated as input data into one of the plurality of image aesthetic comparison models to obtain a first evaluation factor of the first image to be annotated and a second evaluation factor of the second image to be annotated;

[0094] Step S302: based on the first evaluation factor and the second evaluation factor, updating the first basic aesthetic label and the second basic aesthetic label, and using the updated first basic aesthetic label and the second basic aesthetic label as a new first basic aesthetic label and a new second basic aesthetic label, respectively;

[0095] Step S303: using the first image to be annotated and the second image to be annotated as input data and inputting them into other image aesthetic comparison models to obtain a new first evaluation factor and a new second evaluation factor;

[0096] Step S304: updating the new first basic aesthetic label and the new second basic aesthetic label based on the new first evaluation factor and the new second evaluation factor;

[0097] Step S305: Repeat the above steps S303-S304 until all the image aesthetic comparison models are traversed.

[0098] In this embodiment, the purpose of training multiple image aesthetic comparison models is to increase the diversity of image aesthetic comparison models. The differences between these models can be achieved by selecting different training methods and different model complexities. A more accurate result will be obtained by comparing an image pair through multiple comparison models.

[0099] Figure 4 FIG. 1 is a schematic diagram of the main process of a method for constructing a training data set according to another embodiment of the present invention. In this embodiment, it is assumed that there are M images to be annotated (i.e., images to be annotated) (M is an integer greater than or equal to 2), and the loop count threshold is set to K (K is greater than M). The annotation process of the M images to be annotated is as follows: Figure 4 As shown:

[0100] Step S401: Obtaining basic aesthetic labels for each image to be annotated through a basic aesthetic evaluation model;

[0101] Step S402: obtaining multiple groups of image pairs of various categories, and manually annotating each group of image pairs to generate an aesthetic comparison training set;

[0102] Step S403: training the aesthetic comparison training set by using different training methods and different model complexities to obtain N image aesthetic comparison models, where N is an integer greater than 1;

[0103] Step S404: set d=1;

[0104] Step S405: Randomly select two images to be labeled from the M images to be labeled as a first image to be labeled and a second image to be labeled, and determine a first basic aesthetic label of the first image to be labeled and a second basic aesthetic label of the second image to be labeled;

[0105] Step S406: Taking the first image to be labeled and the second image to be labeled as input data, inputting the i-th image aesthetic comparison model to obtain a first evaluation factor of the first image to be labeled and a second evaluation factor of the second image to be labeled, 1 <i≤N;

[0106] Step S407: updating the first basic aesthetic label and the second basic aesthetic label according to the first evaluation factor and the second evaluation factor;

[0107] Step S408: Determine whether i is equal to N, if not, execute step S406, if yes, execute step S409;

[0108] Step S409: determine whether d is greater than or equal to K, if not, execute step S410, if yes, execute S411;

[0109] Step S410: let d=d+1;

[0110] Step S411: normalizing the updated first basic aesthetic label and the second basic aesthetic label, and using the normalized first basic aesthetic label and the second basic aesthetic label as the final aesthetic labels of the first image to be labeled and the second image to be labeled, respectively.

[0111] The method of the embodiment of the present invention uses N image aesthetic comparison models to adjust the aesthetic scores of the images to be annotated. It is a progressive process. Two images whose scores are adjusted by one image aesthetic comparison model are then passed through a second image aesthetic comparison model. In this way, the aesthetic scores of good images will gradually widen the gap with those of bad images. Finally, the scores of some images may be relatively large (for example, greater than 10). At this time, the scores can be normalized so that they fall within a preset range (such as 1-10).

[0112] Figure 5 is a schematic diagram of the main modules of the apparatus 500 for constructing a training data set according to an embodiment of the present invention, such as Figure 5 As shown, the device 500 for constructing a training data set includes:

[0113] A basic label acquisition module 501 is used to randomly select two images to be labeled from the to-be-labeled data set as a first to-be-labeled image and a second to-be-labeled image, and determine a first basic aesthetic label for the first to-be-labeled image and a second basic aesthetic label for the second to-be-labeled image;

[0114] An evaluation factor acquisition module 502 is used to input the first image to be annotated and the second image to be annotated as input data into a pre-trained image aesthetic comparison model to obtain a first evaluation factor of the first image to be annotated and a second evaluation factor of the second image to be annotated;

[0115] The label updating module 503 is used to update the first basic aesthetic label and the second basic aesthetic label according to the first evaluation factor and the second evaluation factor, repeat the above steps until the number of repetitions reaches a threshold, and use the finally obtained basic aesthetic label as the final aesthetic label of the image to be annotated to obtain a training data set.

[0116] Optionally, the basic label acquisition module 501 takes the first image to be labeled and the second image to be labeled as input data respectively, inputs them into a pre-trained basic aesthetic evaluation model, and obtains a first basic aesthetic label for the first image to be labeled and a second basic aesthetic label for the second image to be labeled.

[0117] Optionally, the image aesthetic comparison model is trained according to the following process:

[0118] Obtain multiple sets of image pairs for various purposes, and manually annotate each set of image pairs to generate an aesthetic comparison training set;

[0119] The aesthetic comparison training set is trained to obtain an image aesthetic comparison model.

[0120] Optionally, training the aesthetic comparison training set to obtain an image aesthetic comparison model includes:

[0121] A plurality of training methods and a plurality of model complexities are set; based on the plurality of training methods and the plurality of model complexities, the aesthetic comparison training set is trained to obtain a plurality of image aesthetic comparison models.

[0122] Optionally, the evaluation factor acquisition module 502 is further used to:

[0123] Taking the first image to be labeled and the second image to be labeled as input data, inputting them into one of the multiple image aesthetic comparison models, and obtaining a first evaluation factor of the first image to be labeled and a second evaluation factor of the second image to be labeled;

[0124] The label update module is also used for:

[0125] updating the first basic aesthetic label and the second basic aesthetic label according to the first evaluation factor and the second evaluation factor, and using the updated first basic aesthetic label and the updated second basic aesthetic label as a new first basic aesthetic label and a new second basic aesthetic label, respectively;

[0126] The first image to be annotated and the second image to be annotated are used as input data and input into other image aesthetic comparison models to obtain a new first evaluation factor and a new second evaluation factor;

[0127] Based on the new first evaluation factor and the new second evaluation factor, updating the new first basic aesthetic label and the new second basic aesthetic label;

[0128] The above steps are repeated until the multiple image aesthetic comparison models are traversed.

[0129] Optionally, the label updating module 503 is further configured to update the first basic aesthetic label and the second basic aesthetic label according to the following formula:

[0130]

[0131] Among them, Ra represents the first basic aesthetic label, Rb represents the second basic aesthetic label, Sa represents the first evaluation factor, Sb represents the second evaluation factor, k and β are preset parameters, Ra * Represents the updated first base aesthetic label, Rb * Represents the updated second base aesthetic label.

[0132] Optionally, the label correction module 503 is further used to: normalize the finally obtained basic aesthetic label, and use the normalized basic aesthetic label as the final aesthetic label of the image to be annotated.

[0133] The device for constructing a training data set in an embodiment of the present invention randomly selects two images to be labeled from the data set to be labeled as a first image to be labeled and a second image to be labeled, and determines a first basic aesthetic label of the first image to be labeled and a second basic aesthetic label of the second image to be labeled; uses the first image to be labeled and the second image to be labeled as input data, inputs a pre-trained image aesthetic comparison model, and obtains a first evaluation factor of the first image to be labeled and a second evaluation factor of the second image to be labeled; updates the first basic aesthetic label and the second basic aesthetic label according to the first evaluation factor and the second evaluation factor, repeats the above steps until the number of repetitions reaches a threshold, and uses the finally obtained basic aesthetic label as the final aesthetic label of the image to be labeled to obtain a technical means of the training data set, thereby reducing the complexity and cost of labeling when constructing the training data set and improving scalability.

[0134] The above device can execute the method provided by the embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not described in detail in this embodiment, please refer to the method provided by the embodiment of the present invention.

[0135] Figure 6 An exemplary system architecture 600 is shown to which the method for constructing a training data set or the apparatus for constructing a training data set according to an embodiment of the present invention can be applied.

[0136] like Figure 6 As shown, system architecture 600 may include terminal devices 601, 602, 603, network 604 and server 605. Network 604 is used to provide a medium for communication links between terminal devices 601, 602, 603 and server 605. Network 604 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0137] The user can use the terminal devices 601, 602, 603 to interact with the server 605 through the network 604 to receive or send messages, etc. Various communication client applications can be installed on the terminal devices 601, 602, 603, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0138] The terminal devices 601 , 602 , and 603 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers, etc.

[0139] The server 605 may be a server that provides various services, such as a backend management server that provides support for shopping websites browsed by users using the terminal devices 601, 602, and 603. The backend management server may analyze and process the received data such as product information query requests, and feed back the processing results (such as target push information and product information) to the terminal device.

[0140] It should be noted that the method for constructing a training data set provided in the embodiment of the present invention is generally executed by the server 605 , and accordingly, the device for constructing a training data set is generally disposed in the server 605 .

[0141] It should be understood that Figure 6 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0142] Reference below Figure 7 , which shows a schematic diagram of the structure of a computer system 700 of a terminal device suitable for implementing an embodiment of the present invention. Figure 7 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0143] like Figure 7 As shown, the computer system 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage part 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the system 700 are also stored. The CPU 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0144] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, a mouse, etc.; an output section 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, a modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed, so that a computer program read therefrom is installed into the storage section 708 as needed.

[0145] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the central processing unit (CPU) 701, the above-mentioned functions defined in the system of the present invention are executed.

[0146] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0147] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0148] The modules involved in the embodiments of the present invention may be implemented in software or hardware. The modules described may also be set in a processor. For example, they may be described as: a processor includes a sending module, an acquisition module, a determination module, and a first processing module. The names of these modules do not, in some cases, constitute a limitation on the unit itself. For example, the sending module may also be described as a "module for sending a picture acquisition request to the connected server."

[0149] As another aspect, the present invention further provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes:

[0150] Randomly select two images to be labeled from the to-be-labeled data set as a first to-be-labeled image and a second to-be-labeled image, and determine to obtain a first basic aesthetic label of the first to-be-labeled image and a second basic aesthetic label of the second to-be-labeled image;

[0151] The first image to be annotated and the second image to be annotated are used as input data and input into a pre-trained image aesthetic comparison model to obtain a first evaluation factor of the first image to be annotated and a second evaluation factor of the second image to be annotated;

[0152] modifying and updating the first basic aesthetic label and the second basic aesthetic label according to the first evaluation factor and the second evaluation factor;

[0153] Repeat the above steps until the number of repetitions reaches a threshold, and use the final basic aesthetic label as the final aesthetic label of the image to be labeled to obtain a training data set.

[0154] The technical solution of the embodiment of the present invention reduces the complexity and cost of annotation when constructing a training data set and improves scalability.

[0155] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for constructing a training data set, characterized in that: include: Randomly selecting two images to be labeled from a dataset to be labeled as a first image to be labeled and a second image to be labeled, and determining a first basic aesthetic label of the first image to be labeled and a second basic aesthetic label of the second image to be labeled, including: using the first image to be labeled and the second image to be labeled as input data, respectively, inputting the pre-trained basic aesthetic evaluation model, and obtaining the first basic aesthetic label of the first image to be labeled and the second basic aesthetic label of the second image to be labeled; The first image to be annotated and the second image to be annotated are used as input data and input into a pre-trained image aesthetic comparison model to obtain a first evaluation factor of the first image to be annotated and a second evaluation factor of the second image to be annotated; wherein a plurality of groups of image pairs of various categories are obtained, and each group of image pairs is manually annotated to generate an aesthetic comparison training set; the aesthetic comparison training set is trained to obtain an image aesthetic comparison model; updating the first basic aesthetic label and the second basic aesthetic label according to the first evaluation factor and the second evaluation factor; Repeat the above steps until the number of repetitions reaches a threshold, and use the final basic aesthetic label as the final aesthetic label of the image to be labeled to obtain a training data set.

2. The method according to claim 1, characterized in that The aesthetic comparison training set is trained to obtain an image aesthetic comparison model including: A plurality of training methods and a plurality of model complexities are set; based on the plurality of training methods and the plurality of model complexities, the aesthetic comparison training set is trained to obtain a plurality of image aesthetic comparison models.

3. The method according to claim 2, characterized in that The first image to be annotated and the second image to be annotated are used as input data and input into a pre-trained image aesthetic comparison model to obtain a first evaluation factor of the first image to be annotated and a second evaluation factor of the second image to be annotated, including: Taking the first image to be labeled and the second image to be labeled as input data, inputting them into one of the multiple image aesthetic comparison models, and obtaining a first evaluation factor of the first image to be labeled and a second evaluation factor of the second image to be labeled; Updating the first basic aesthetic label and the second basic aesthetic label according to the first evaluation factor and the second evaluation factor includes: updating the first basic aesthetic label and the second basic aesthetic label according to the first evaluation factor and the second evaluation factor, and using the updated first basic aesthetic label and the updated second basic aesthetic label as a new first basic aesthetic label and a new second basic aesthetic label, respectively; The first image to be annotated and the second image to be annotated are used as input data and input into other image aesthetic comparison models to obtain a new first evaluation factor and a new second evaluation factor; Based on the new first evaluation factor and the new second evaluation factor, updating the new first basic aesthetic label and the new second basic aesthetic label; The above steps are repeated until the multiple image aesthetic comparison models are traversed.

4. The method according to any one of claims 1 to 3, characterized in that: The first basic aesthetic label and the second basic aesthetic label are updated according to the following formula: Among them, Ra represents the first basic aesthetic label, Rb represents the second basic aesthetic label, Sa represents the first evaluation factor, Sb represents the second evaluation factor, k and β are preset parameters, Ra * Represents the updated first base aesthetic label, Rb * Represents the updated second base aesthetic label.

5. The method according to claim 4, characterized in that The final basic aesthetic labels are used as the final aesthetic labels of the image to be annotated, including: The final basic aesthetic labels are normalized and used as the final aesthetic labels of the images to be labeled.

6. A device for constructing a training data set, characterized in that: include: A basic label acquisition module is used to randomly select two images to be labeled from the to-be-labeled data set as a first to-be-labeled image and a second to-be-labeled image, and determine a first basic aesthetic label for the first to-be-labeled image and a second basic aesthetic label for the second to-be-labeled image, including: taking the first to-be-labeled image and the second to-be-labeled image as input data, respectively, and inputting the pre-trained basic aesthetic evaluation model to obtain the first basic aesthetic label for the first to-be-labeled image and the second basic aesthetic label for the second to-be-labeled image; An evaluation factor acquisition module is used to input the first image to be annotated and the second image to be annotated as input data into a pre-trained image aesthetic comparison model to obtain a first evaluation factor of the first image to be annotated and a second evaluation factor of the second image to be annotated; wherein a plurality of groups of image pairs of various categories are obtained, and each group of image pairs is manually annotated to generate an aesthetic comparison training set; and the aesthetic comparison training set is trained to obtain an image aesthetic comparison model; A label updating module is used to update the first basic aesthetic label and the second basic aesthetic label according to the first evaluation factor and the second evaluation factor, repeat the above steps until the number of repetitions reaches a threshold, and use the finally obtained basic aesthetic label as the final aesthetic label of the image to be annotated to obtain a training data set.

7. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 5.

8. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images