Picture labeling method and device, storage medium and electronic device

Abstract

This application provides an image annotation method, apparatus, storage medium, and electronic device. The method includes: inputting an image to be annotated into an image classification model and an optical character recognition model, wherein the image classification model is trained using a set of sample images, and the set of sample images is obtained from an image library based on target keywords determined by a set of preset labels; obtaining a set of image tags for the image to be annotated output by the image classification model, and obtaining image text for the image to be annotated output by the optical character recognition model, wherein the set of image tags includes a set of image style tags and a set of image theme tags, the set of image style tags includes primary image style tags and secondary image style tags, and the set of image theme tags includes primary image theme tags and secondary image theme tags; and determining the target tags corresponding to the image to be annotated based on the set of image tags and image text, thereby determining the annotation result of the image to be annotated.

Description

[Technical Field]

[0001] This application relates to the field of image processing technology, and more specifically, to an image annotation method and apparatus, storage medium, and electronic device. [Background Technology]

[0002] In the era of digital economy, data has become an important factor of production, and the digital transformation of the economy and society has entered an accelerated period. In order to better use big data applications to help business development, digital content tags are linked with intelligent recommendations and application scenarios to realize the conversion of traffic value. Image labeling and classification are also becoming more important in the banking and other financial industries.

[0003] However, existing image annotation processes require significant manual intervention. Due to human factors, both the training of image annotation models and the acquisition of annotation results take considerable time. Therefore, existing image annotation methods suffer from low annotation efficiency. [Summary of the Invention]

[0004] The purpose of this application is to provide an image annotation method, apparatus, storage medium, and electronic device to at least solve the problem of low annotation efficiency in related image annotation methods.

[0005] According to one aspect of the embodiments of this application, an image annotation method is provided, comprising: inputting an image to be annotated into an image classification model and an optical character recognition model, wherein the image classification model is a model obtained by training an image classification model to be trained using a set of sample images, the set of sample images being images obtained from an image library based on target keywords, and the target keywords being keywords determined based on a set of preset tags; obtaining a set of image tags output by the image classification model corresponding to the image to be annotated, and obtaining image text output by the optical character recognition model corresponding to the image to be annotated, wherein the set of image tags includes a set of image style tags and a set of image theme tags, the set of image style tags including primary image style tags and secondary image style tags, and the set of image theme tags including primary image theme tags and secondary image theme tags; determining a target tag corresponding to the image to be annotated based on the set of image tags and the image text; and determining an annotation result for annotating the image to be annotated based on the target tag.

[0006] According to another aspect of the embodiments of this application, an image annotation apparatus is also provided, comprising: an input unit, configured to input an image to be annotated into an image classification model and an optical character recognition model, wherein the image classification model is a model obtained by training an image classification model to be trained using a set of sample images, the set of sample images being images obtained from an image library based on target keywords, and the target keywords being keywords determined based on a set of preset tags; an acquisition unit, configured to acquire a set of image tags output by the image classification model corresponding to the image to be annotated, and to acquire image text output by the optical character recognition model corresponding to the image to be annotated, wherein the set of image tags includes a set of image style tags and a set of image theme tags, the set of image style tags including primary image style tags and secondary image style tags, and the set of image theme tags including primary image theme tags and secondary image theme tags; a first determination unit, configured to determine a target tag corresponding to the image to be annotated based on the set of image tags and the image text; and a second determination unit, configured to determine an annotation result for the image to be annotated based on the target tag.

[0007] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer-readable storage medium, and the computer program is configured to execute the above-described image annotation method at runtime.

[0008] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the above-described image annotation method through the computer program.

[0009] In this embodiment, the image annotation is completed by labeling images from the dimensions of style and theme, supplemented by image text recognition results. The image to be annotated is input into an image classification model and an optical character recognition model. The image classification model is a model trained using a set of sample images. The set of sample images are images obtained from an image library based on target keywords, and the target keywords are keywords determined based on a set of preset tags. The image labels corresponding to the image to be annotated are obtained from the output of the image classification model, and the image text corresponding to the image to be annotated is obtained from the output of the optical character recognition model. The set of image labels includes a set of image style tags and a set of image theme tags. The system includes primary and secondary image style tags, and a set of image theme tags, including primary and secondary image theme tags. Based on these tags and the image text, target tags corresponding to the images to be labeled are determined. Based on the target tags, the labeling results for the images to be labeled are determined. An image classification model automatically identifies the tags containing theme and style for the images to be labeled, and an optical character recognition model is used to recognize the text in the images to compensate for the limitations of image classification models in recognizing image text. This improves the accuracy of image labeling without requiring human intervention, thus increasing labeling efficiency and solving the problem of low labeling efficiency in related image labeling methods. [Attached Image Description]

[0010] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0011] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a schematic diagram of the hardware environment for an optional image annotation method according to an embodiment of this application;

[0013] Figure 2 This is a flowchart illustrating an optional image annotation method according to an embodiment of this application;

[0014] Figure 3 This is a schematic diagram of an optional image annotation method according to an embodiment of this application;

[0015] Figure 4This is a flowchart illustrating another optional image annotation method according to an embodiment of this application;

[0016] Figure 5 This is a flowchart illustrating another optional image annotation method according to an embodiment of this application;

[0017] Figure 6 This is a flowchart illustrating another optional image annotation method according to an embodiment of this application;

[0018] Figure 7 This is a structural block diagram of an optional image annotation device according to an embodiment of this application;

[0019] Figure 8 This is a structural block diagram of an optional electronic device according to an embodiment of this application.

Detailed Implementation Methods

[0020] The present application will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of the present application can be combined with each other.

[0021] It should be noted that the terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0022] According to one aspect of the embodiments of this application, an image annotation method is provided. Optionally, in this embodiment, the above-described image annotation method can be applied to, for example... Figure 1 The hardware environment shown consists of terminal device 102 and server 104. For example... Figure 1 As shown, server 104 is connected to terminal device 102 via a network and can be used to provide services (such as application services) to the terminal or clients installed on the terminal. A database can be set up on the server or independently of the server to provide data storage services for server 104. Cloud computing and / or edge computing services can be configured on the server or independently of the server to provide data processing services for server 104.

[0023] The aforementioned network may include, but is not limited to, at least one of the following: wired network, wireless network. The aforementioned wired network may include, but is not limited to, at least one of the following: wide area network, metropolitan area network, local area network. The aforementioned wireless network may include, but is not limited to, at least one of the following: Wi-Fi, Bluetooth. The terminal device 102 may not be limited to smart devices such as PCs, mobile phones, and tablets.

[0024] The image annotation method of this application embodiment can be executed by server 104, or it can be jointly executed by server 104 and terminal device 102. Taking the image annotation method of this embodiment executed by server 104 as an example, Figure 2 This is a flowchart illustrating an optional image annotation method according to an embodiment of this application, such as... Figure 2 As shown, the process of this method may include the following steps:

[0025] Step S202: Input the image to be labeled into the image classification model and the optical character recognition model. The image classification model is a model obtained by training the image classification model to be trained using a set of sample images. The set of sample images are images obtained from the image library based on the target keywords. The target keywords are keywords determined based on a set of preset labels.

[0026] The image annotation method in this embodiment can be applied to scenarios where images are annotated. Image annotation refers to determining the tags to which an image belongs and batch classifying image materials according to the tags to facilitate subsequent use of each image. Image tags can include the image's theme, style, etc.

[0027] Current image annotation processes require significant manual intervention. Due to human factors, training the image annotation model and obtaining the annotation results take considerable time, resulting in low efficiency. Furthermore, because the annotation process relies heavily on manual intervention, the results are subjective, prone to errors, and have low accuracy.

[0028] To at least address some of the aforementioned issues, this embodiment employs image classification models and optical character recognition models for intelligent annotation of the images to be labeled, thereby reducing the impact of human factors during the annotation process, thus improving annotation efficiency and the accuracy of the annotation results.

[0029] The image classification model described above can be a model obtained by training a training image classification model using a set of sample images, and can be a neural network model. Considering that traditional Convolutional Neural Network (CNN) models have locality of reference in convolution operations, resulting in low accuracy, the image classification model in this embodiment can be a Swin Transformer (a neural network model used for computer vision and natural language processing tasks). Internal model fusion can be achieved through ten-cropping (a processing step in the transform model). A Transformer is a model composed of an encoder and a decoder. The encoder encodes the input data, and the decoder decodes the encoded data representation to obtain the output. Transformers are based on multi-head attention mechanisms, have good memory capabilities, and can remember long-range information. Figure 3As shown, the Swin Transformer, based on the Transformer, employs a sliding window operation and a hierarchical design. The sliding window operation restricts attention computation to a single window, which on the one hand introduces the locality of convolution operations in CNN models, and on the other hand saves computational resources. The Swin Transformer includes two self-attention structures: W-MSA (Window-Multihead Self-Attention) and SW-MSA (Shifted Window-Multihead Self-Attention). W-MSA divides an image evenly into multiple windows, while SW-MSA performs the transformer operation within a small window. In two consecutive Swin Transformer Blocks (i.e., Swin Transformer modules), the image is processed through LN (Layer Normalization), MLP (Multi-Layer Perceptron), and the two self-attention structures mentioned above. The SwinTransformer's hierarchical design comprises four stages. Stage 1 includes a linear embedding layer and a SwinTransformer module. Stages 2, 3, and 4 each contain a tile merging layer and a SwinTransformer module. Each stage reduces the resolution of the input feature map, expanding the receptive field layer by layer, similar to a CNN, to process information at different scales. The optical character recognition model can be an OCR (Optical Character Recognition) model. Optical character recognition refers to the process of analyzing and recognizing textual data in image files to obtain text and layout information; that is, recognizing the text in an image and returning it as text. A set of sample images can be images obtained from an image library based on target keywords. Target keywords can be keywords determined based on a set of preset labels.

[0030] It should be noted that the image classification model described above can be trained in an environment with a GPU (Graphics Processing Unit). By reducing the training time and increasing the number of iterations for model optimization, the training efficiency can be improved while simultaneously increasing the model's accuracy in recognizing image labels. The trained and optimized model is then uploaded to a CPU (Central Processing Unit) environment and tested with test images to determine its effectiveness.

[0031] Step S204: Obtain a set of image labels output by the image classification model corresponding to the image to be labeled, and obtain the image text output by the optical character recognition model corresponding to the image to be labeled. The set of image labels includes a set of image style labels and a set of image theme labels. The set of image style labels includes primary image style labels and secondary image style labels. The set of image theme labels includes primary image theme labels and secondary image theme labels.

[0032] To enable labeled images to be used in more scenarios and fields, the image labels obtained from the output of the image classification model can contain multiple labels, and each label can have multiple levels of labels, such as first-level and second-level labels.

[0033] In this embodiment, a set of image labels corresponding to the image to be labeled, output by the image classification model, can be obtained. A set of image labels may include a set of image style labels and a set of image theme labels. A set of image style labels may include primary image style labels and secondary image style labels. A set of image theme labels may include primary image theme labels and secondary image theme labels. Secondary labels can be labels that describe the details in the image to be labeled in more detail than primary labels. As shown in Table 1, secondary labels can be labels that better reflect the details of the image than primary labels. For example, primary style labels can be phrases such as illustration or photography that reflect the overall style of the image to be labeled, while secondary style labels can be phrases such as cartoon doodle or science fiction that further reflect the details within the style of the image. Primary theme labels can be phrases such as festival, scene, or icon background that reflect the overall theme of the image to be labeled, while secondary theme labels can be phrases that further reflect the details within the theme of the image. For example, if the primary theme label of an image is "festival," its secondary theme label can be a specific festival, such as "Lantern Festival." It should be noted that the labels in Table 1 are just examples and do not mean that the image classification model can only identify two primary style labels from the image, nor does it mean that the image classification model can only identify four primary topic labels from the image.

[0034] Table 1

[0035]

[0036] It should be noted that the primary and secondary labels that the image classification model in this embodiment can output are not limited to the labels in Table 1. Table 1 is only an optional example.

[0037] An image can have at least two labels: a primary style label and a primary topic label. Secondary labels, which are more granular than primary labels, are output based on the image's clarity and the model's recognition ability. When an image is classified using an image classification model, an image can have one or more secondary style labels and one or more secondary topic labels simultaneously, or it can have only one of the two, or it can have no secondary labels at all (i.e., neither secondary style nor secondary topic labels).

[0038] For the image to be labeled, the corresponding image text output by the optical character recognition (OCR) model can also be obtained. It should be noted that the image text output by the OCR model can be text recognized from the image to be labeled. If there is no text in the image to be labeled, the OCR model cannot recognize the image text.

[0039] Step S206: Based on a set of image labels and image text, determine the target label corresponding to the image to be labeled.

[0040] In this embodiment, based on a set of image tags and image text obtained, a target tag corresponding to the image to be labeled can be determined. The target tag may include one or more image tags from a set of image tags and image text.

[0041] It should be noted that the above set of image labels can be all the image labels output by the image classification model corresponding to the image to be labeled, but not all of these image labels are closely related to the image to be labeled.

[0042] Optionally, for the acquired image text, the image text can be used together with one or more image tags from a set of acquired image tags as the target tag for the image to be labeled. Alternatively, one or more image tags from a set of acquired image tags can be corrected or verified based on the image text. For example, if the image text is completely opposite to some image tags in a set of acquired image tags, those individual image tags can be deleted, or the image to be labeled can be re-identified based on those individual image tags.

[0043] Step S208: Determine the annotation results for the image to be annotated based on the target label.

[0044] Based on the target labels, the annotation results for the image to be labeled can be determined. The target labels can be directly used as the annotation results for the image to be labeled, and the annotation results can be recorded and saved. Considering that different use cases for the images to be labeled may require each label to correspond to a recognition standard, the target labels and the recognition standards corresponding to the target labels can also be saved together. The recognition standards here can be the degree of correlation between each label and the image to be labeled, generated by the image classification model when determining each image label, or other recognition standards, which will not be elaborated on in this embodiment.

[0045] Through steps S202 to S208, the image to be labeled is input into an image classification model and an optical character recognition model. The image classification model is a model trained using a set of sample images. The sample images are images obtained from an image library based on target keywords, and the target keywords are keywords determined based on a set of preset tags. A set of image tags corresponding to the image to be labeled, output by the image classification model, and image text corresponding to the image to be labeled, output by the optical character recognition model, are obtained. The set of image tags includes a set of image style tags and a set of image theme tags. The set of image style tags includes primary image style tags and secondary image style tags, and the set of image theme tags includes primary image theme tags and secondary image theme tags. Based on the set of image tags and image text, the target tag corresponding to the image to be labeled is determined. Based on the target tag, the labeling result for the image to be labeled is determined. This solves the problem of low labeling efficiency in related image labeling methods and improves labeling efficiency.

[0046] In one exemplary embodiment, determining the target label corresponding to the image to be labeled based on a set of image labels and image text includes:

[0047] S11, based on the probability of each image tag in a set of image tags, select the image style tag with the highest probability and greater than or equal to the first preset probability threshold from a set of image style tags as the target style tag, and select the image theme tag with the highest probability and greater than or equal to the second preset probability threshold from a set of image theme tags as the target theme tag.

[0048] S12, if the image text conforms to the preset regular expression rules, the image text is identified as the target recognition label;

[0049] The target tags include target style tags, target theme tags, and target identification tags.

[0050] The correlation between each image tag in the aforementioned set of image tags and the image to be labeled can be represented in the form of probability; the higher the probability, the greater the correlation. In this embodiment, when obtaining a set of image tags through an image classification model, the image style tag with the highest probability (greater than or equal to a first preset probability threshold) can be selected as the target style tag from a set of image style tags, based on the probability corresponding to each image tag in the set. Similarly, the image theme tag with the highest probability (greater than or equal to a second preset probability threshold) can be selected as the target theme tag from a set of image theme tags.

[0051] It should be noted that the image classification model uses an end-to-end direct label prediction approach. The model output can be the probability corresponding to each label. Image labels are filtered and multi-label classification is achieved by setting probability thresholds for each label (labels with probabilities greater than the threshold are considered correct). The label thresholds can be flexibly adjusted based on subsequent data. If the probability corresponding to each image style label in a set of image style labels is less than the first preset probability threshold, then no image style label is needed. Similarly, if the probability corresponding to each image theme label in a set of image subject labels is less than the second preset probability threshold, then no image style label is needed.

[0052] For the acquired image text, a regular expression rule can be preset. If the output image text meets the preset regular expression rule, the image text will be identified as the target recognition tag.

[0053] Optionally, if the output image text does not meet the preset regular expression rules, the image text is discarded, and only the obtained set of image tags is used as the target tags for the image to be labeled.

[0054] It should be noted that the aforementioned optical character recognition model can be an OCR interface. By calling a customized OCR interface, the regular expression rules at the output of the OCR model can be enriched, or it can be fused into the aforementioned image classification model in the form of embeddings (vectors). If the recognized text in the image meets the regular expression rules (for example, the text contains the word "Spring Festival"), the image is directly assigned the corresponding label according to the rules.

[0055] like Figure 4 As shown, for an image to be labeled, the image classification model and OCR model can respectively obtain the identified label, the probability corresponding to the label, and the identified text in the image. The label corresponding to the image is obtained according to the label probability threshold and regular expression rules.

[0056] Optionally, the above-mentioned determination of the annotation result for the image to be annotated based on the target label can also be achieved in the following ways:

[0057] S21, determine the probability of the target label corresponding to the target label as the annotation result for the image to be annotated; or,

[0058] S22, the target label is determined as the annotation result of the image to be annotated.

[0059] When determining the probability of the target label and its corresponding value as the annotation result for the image to be annotated, an Excel (spreadsheet) or CSV (Comma-Separated Values, a type of plain text used to store data) file can be generated, as shown in Table 2. The file records the URL (Uniform Resource Locator) of each image, the image name, the first-level style tag, the highest probability of the first-level style, the second-level style tag, the highest probability of the second-level style, the first-level topic tag, the highest probability of the first-level topic, the second-level topic tag, and the highest probability of the second-level topic.

[0060] Table 2

[0061]

[0062] Table 2 (continued)

[0063]

[0064] When determining the target labels as the annotation results for the image to be labeled, the labels obtained by the model can be added to the image filename. The naming rule for the filename can be: primary style label & secondary style label & primary theme label & secondary theme label @ original image filename. There may be multiple secondary style, primary theme, and secondary theme labels, separated by "-", for example, the secondary theme is "Spring Festival - Food". If the probability of a certain type of label predicted for the image is less than a threshold, then the label for that type of label is empty. For example, in "Illustration & Cartoon Doodle & Festival - Icon Background & @image.jpeg", the secondary theme label is empty.

[0065] This embodiment improves the accuracy of image annotation by determining the image annotation results from the model output based on a pre-set probability threshold and regular expression rules.

[0066] In one exemplary embodiment, before inputting the image to be labeled into the image classification model and the optical character recognition model, the above method further includes:

[0067] S31, Select target keywords from a set of preset tags, and obtain a set of candidate images from the image library based on the target keywords. The set of preset tags includes a set of theme tags and a set of style tags, and each image in the candidate image set is associated with the target keyword.

[0068] S32, Based on the candidate image set, determine a set of sample images;

[0069] S33. A set of sample images is used to train the image style recognition model and the image subject recognition model, respectively, to obtain the image style recognition model and the image subject recognition model. The image classification model includes the image style recognition model and the image subject recognition model, and the image classification model includes the image style recognition model and the image subject recognition model.

[0070] For training an image classification model, images with known image labels can be used. These images, with known labels, can be automatically collected web images using pre-designed keywords. In this embodiment, a target keyword can be selected from a set of preset labels, and a candidate image set can be obtained from an image library based on the target keyword. The set of preset labels can be pre-defined tags, including a set of topic tags and a set of style tags. Each image in the obtained candidate image set is associated with the target keyword.

[0071] Based on the obtained candidate image set, the candidate images can be processed to adjust them into images that can be used for model training, resulting in a set of sample images.

[0072] Since image style tags and image topic tags are two different types of tags, to improve the accuracy of image annotation, the aforementioned image classification model can include an image style recognition model and an image topic recognition model. In this embodiment, a set of sample images can be used to train the image style recognition model and the image topic recognition model to be trained, respectively, to obtain the image style recognition model and the image topic recognition model. Correspondingly, the aforementioned image classification model to be trained includes the image style recognition model and the image topic recognition model to be trained.

[0073] Based on pre-designed keywords, these keywords can be used as code input to automatically collect online images during code execution. Furthermore, the collected images can be automatically organized and saved in folders with corresponding tags. This allows the model to directly use these images for prediction. During image collection, images can be downloaded using Python's requests library (a cross-platform computer programming language) based on the obtained image URLs. Additionally, sample images used for model training can be pre-downloaded, labeled image sets.

[0074] In this embodiment, sample images for model training are obtained based on keywords selected from preset styles and topic tags, which can improve the accuracy of the model in determining the style and topic tags of images.

[0075] In one exemplary embodiment, selecting target keywords from a set of preset tags includes:

[0076] S41, Select one preset tag from a set of preset tags as the target keyword; or...

[0077] S42, combine at least two preset tags from a set of preset tags into a target keyword; or...

[0078] S43, combine at least one preset tag from a set of preset tags with other keywords to form a target keyword, wherein the other keywords are tags of other types besides a set of topic tags and a set of style tags.

[0079] When selecting a target keyword from a set of preset tags, you can choose one preset tag as the target keyword, that is, use a single tag from the set of preset tags as the target keyword to search for corresponding images in the image library. Alternatively, you can combine at least two preset tags from the set to form a target keyword, or combine at least one preset tag from the set with other keywords to form a target keyword.

[0080] The above-mentioned combination of at least two preset tags from a set of preset tags into a target keyword can be either combining at least two preset tags with different themes or styles from a set of preset tags, or combining one or more theme tags with one or more style tags from a set of preset tags.

[0081] When combining at least one preset tag from a set of preset tags with other keywords to form a target keyword, the other keywords can be phrases unrelated to the style tags and theme tags in the set of preset tags.

[0082] For example, taking "direct tag search," "tag combination search," and "tag-related keyword search" as keyword generation methods, direct tag search means using a single tag as a keyword to search for corresponding images in the image library. Tag combination search means using a combination of style and theme tags, or different theme tags, as keywords (e.g., "Spring Festival" and "food") to search for images with multiple tags in the image library. Tag-related keyword search involves manually setting keywords related to tags and tag combinations (e.g., "traditional" and "architecture") as keywords for image searches.

[0083] By determining the keywords of the acquired sample images in different ways through this embodiment, the diversity of the acquired sample images can be improved, thereby improving the accuracy of the trained image classification model in recognizing image labels.

[0084] In one exemplary embodiment, a set of sample images is determined based on a candidate image set, including:

[0085] S51, Based on the relevance of each image in the candidate image set to the target keyword, select a set of candidate images from the candidate image set, wherein the relevance of each image in the set of candidate images to the target keyword is greater than or equal to a preset threshold.

[0086] S52, adjust the image size of each image in a set of candidate images to a preset size to obtain a set of sample images.

[0087] Considering that some images in the candidate image set retrieved based on the target keyword may contain a large amount of noisy data, meaning that the actual labels of some images may not match the target keyword, this embodiment can filter a group of candidate images from the candidate image set based on the relevance of each image in the candidate image set to the target keyword. Here, the relevance of each image in the group of candidate images to the target keyword can be greater than or equal to a preset threshold.

[0088] The above-mentioned screening process can be completed through manual filtering, code filtering, or a combination of both. This embodiment does not limit this process.

[0089] Considering that some images in the collected candidate image set may have high resolution and many pixels, resulting in a large computational burden when training the image classification model (especially in the calculation process based on global self-attention), preprocessing can be performed on the selected candidate images to reduce the computational burden of model training. Preprocessing can involve adjusting the image size of each image in a set of candidate images to a preset size, i.e., cropping and resizing each image in a set of candidate images to a uniform size, to obtain a set of sample images.

[0090] Optionally, to improve the accuracy of image classification models in recognizing image labels, sample images can be rotated at different angles to obtain multi-angle sample images corresponding to the original sample images. For example... Figure 5 As shown, the images used to train the model can go through an image acquisition process, an image selection process, and an image processing process.

[0091] Optionally, the obtained set of sample images can be divided into two parts: one part is used as training samples for model training, and the other part is used as test samples for testing the trained model.

[0092] In this embodiment, filtering searched images based on their relevance to keywords can improve the accuracy of image tag recognition by the model trained on those images. In addition, adjusting the size of the sample images can reduce the computational load during model training.

[0093] In one exemplary embodiment, a set of sample images is used to train a model for image style recognition and a model for image subject recognition, respectively, to obtain the image style recognition model and the image subject recognition model, including:

[0094] S61, a set of sample images and corresponding actual image style labels are used to train the image style recognition model until the first target loss value of the image style recognition model meets the first preset convergence condition. The training ends, and the image style recognition model at the end of the training is determined as the image style recognition model. The first target loss value is the loss value between the image style label determined by the image style recognition model based on the input image and the actual image style label corresponding to the input image. The image style recognition model is used to determine the secondary image style label of the image to be labeled, and the parent image style label of the secondary image style label of the image to be labeled is determined as the primary image style label of the image to be labeled.

[0095] S62, using a set of sample images and corresponding actual image topic labels, train the image topic recognition model to be trained until the second target loss value of the image topic recognition model meets the second preset convergence condition, then end the training, and determine the image topic recognition model at the end of the training as the image topic recognition model. Here, the second target loss value is the loss value between the image topic label determined by the image topic recognition model based on the input image to be trained and the actual image topic label corresponding to the input image to be trained. The image topic recognition model is used to determine the secondary image topic label of the image to be labeled, and the parent image topic label of the secondary image topic label of the image to be labeled is determined as the primary image topic label of the image to be labeled.

[0096] Considering the correlation between secondary style tags and primary style tags, and between secondary topic tags and primary topic tags, to improve the training efficiency of the model, this embodiment uses a set of sample images and corresponding actual image style tags to train the image style recognition model until the first target loss value of the image style recognition model meets the first preset convergence condition. Training then ends, and the image style recognition model at the end of training is determined as the image style recognition model. The trained image style recognition model can be used to determine the secondary image style tags of the images to be labeled, and the parent image style tag of the secondary image style tag of the image to be labeled is determined as the primary image style tag of the image to be labeled.

[0097] The parent image style tag of the second-level image style tag of the above-mentioned image to be labeled can be mapped from the identified second-level image style tags.

[0098] Optionally, the image style recognition model trained above can also be used only to determine the secondary image style labels of the image to be labeled. Other models then determine the parent image style label of the secondary image style label of the image to be labeled as the primary image style label.

[0099] The aforementioned first target loss value can be the loss value between the image style recognition model under test, which determines the image style label based on the input image under test, and the actual image style label corresponding to the input image under test.

[0100] Alternatively, the same method used to train the model for determining secondary image style labels can be employed to train the model for determining secondary image topic labels. That is, a set of sample images and their corresponding actual image topic labels are used to train the image topic recognition model until the second objective loss value of the model satisfies the second preset convergence condition. Training then ends, and the model at the end of training is identified as the image topic recognition model. Here, the second objective loss value can be the loss between the image topic labels determined by the model based on the input image and the actual image topic labels corresponding to the input image. The image topic recognition model can be used to determine the secondary image topic labels of the image to be labeled, and the parent image topic labels of the secondary image topic labels are identified as the primary image topic labels of the image to be labeled. Alternatively, the image topic recognition model can be used only to determine the secondary image topic labels of the image to be labeled, while other models determine the parent image topic labels of the secondary image topic labels as the primary image topic labels of the image to be labeled. The parent image topic tags of the secondary image topic tags of the image to be labeled can also be mapped from the identified secondary image topic tags.

[0101] Alternatively, a set of sample images and corresponding actual image style labels and actual image theme labels can be used to train a model that can simultaneously determine secondary image style labels and secondary image theme labels. Or, a model can be trained that can simultaneously determine secondary image style labels and secondary image theme labels, and determine the parent image theme label of the secondary image theme label of the image to be labeled and the parent image style label of the secondary image style label as the primary image theme label and primary image style label of the image to be labeled, respectively.

[0102] This embodiment improves the efficiency of model training by training a model for recognizing secondary labels in images using sample images.

[0103] In one exemplary embodiment, a set of sample images is used to train the image style recognition model and the image subject recognition model to be trained, respectively, to obtain the image style recognition model and the image subject recognition model, and the method further includes:

[0104] S71, using a set of sample images and the actual first-level image style labels corresponding to the set of sample images, train the first image style recognition model to be trained, and obtain the first image style recognition model.

[0105] S72, using a set of sample images and the actual secondary image style labels corresponding to the set of sample images, the second image style recognition model is trained to obtain the second image style recognition model;

[0106] S73, based on a set of sample images and the actual first-level image topic labels corresponding to the set of sample images, train the first image topic recognition model to be trained to obtain the first image topic recognition model;

[0107] S74, based on a set of sample images and the actual secondary image topic labels corresponding to the set of sample images, train the second image topic recognition model to obtain the second image topic recognition model;

[0108] The image style recognition model includes a first image style recognition model and a second image style recognition model, and the image theme recognition model includes a first image theme recognition model and a second image theme recognition model. The first image style recognition model is used to determine the primary image style label of the image to be labeled, and the second image style recognition model is used to determine the secondary image style label of the image to be labeled. The first image theme recognition model is used to determine the primary image theme label of the image to be labeled, and the second image theme recognition model is used to determine the secondary image theme label of the image to be labeled.

[0109] To improve the accuracy of the trained model in recognizing image labels, in this embodiment, a set of sample images and the corresponding actual first-level image style labels, actual second-level image style labels, actual first-level image theme labels, and actual second-level image theme labels can be used to train the model, so as to obtain the first image style recognition model, the second image style recognition model, the first image theme recognition model, and the second image theme recognition model, respectively.

[0110] The training process of the first image style recognition model, the second image style recognition model, the first image theme recognition model, and the second image theme recognition model can be similar to the model training process in the aforementioned embodiments. That is, a set of sample images and the actual image labels corresponding to the set of sample images are used to train the model until the loss value meets a certain preset convergence condition, and then the training ends.

[0111] Alternatively, a set of sample images and the corresponding primary image style labels, secondary image style labels, primary image theme labels, and secondary image theme labels can be used to train a model that can simultaneously determine the primary image style labels, secondary image style labels, primary image theme labels, and secondary image theme labels.

[0112] like Figure 6As shown, before using an image classification model, the process may include defining an analysis framework, image acquisition, data processing, training the image classification model, prediction using an image ensemble model, model deployment, and application. The image ensemble model can be a composite model that includes multiple models with different functions as described in the preceding embodiments.

[0113] In this embodiment, four models are trained based on the actual primary image style tags, actual secondary image style tags, actual primary image theme tags, and actual secondary image theme tags of the sample images. These models are used to determine the primary image style tags, secondary image style tags, primary image theme tags, and secondary image theme tags, respectively. This can improve the accuracy of the models in recognizing image tags.

[0114] The image annotation method in the embodiments of this application will be explained below with reference to optional examples. In this optional example, the steam cleaning equipment is a steam floor scrubber, and the heating component is a heat pump.

[0115] This optional example provides a method and related apparatus for intelligent annotation of static images based on style and theme dimensions. It annotates images based on style and theme dimensions, and combines an OCR model to recognize text in the images to compensate for the limitations of classification models in recognizing text in images. This not only achieves automatic image annotation, but also improves the accuracy of image annotation.

[0116] The image annotation method in this optional example may include the following steps:

[0117] Step 1: Combine keywords with tags to obtain sample images and train the image classification model.

[0118] Step 2: Recognize the image to be labeled using an image classification model and an OCR model.

[0119] Step 3: Based on the label results output by the image classification model and the text results output by the OCR, determine the label corresponding to the image to be labeled by using the label probability threshold and regular expression rules.

[0120] This example can improve the efficiency and accuracy of image annotation.

[0121] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0122] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM (Read-Only Memory) / RAM (Random Access Memory), magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0123] According to another aspect of the embodiments of this application, an image annotation apparatus for implementing the above-described image annotation method is also provided. Figure 7 This is a structural block diagram of an optional image annotation device according to an embodiment of this application, such as... Figure 7 As shown, the device may include:

[0124] The input unit 702 is used to input the image to be labeled into the image classification model and the optical character recognition model. The image classification model is a model obtained by training the image classification model to be trained using a set of sample images. The set of sample images are images obtained from the image library according to the target keywords. The target keywords are keywords determined according to a set of preset labels.

[0125] The acquisition unit 704 is connected to the input unit 702 and is used to acquire a set of image labels corresponding to the image to be labeled output by the image classification model, and to acquire the image text corresponding to the image to be labeled output by the optical character recognition model. The set of image labels includes a set of image style labels and a set of image theme labels. The set of image style labels includes a first-level image style label and a second-level image style label. The set of image theme labels includes a first-level image theme label and a second-level image theme label.

[0126] The first determining unit 706, connected to the obtaining unit 704, is used to determine the target label corresponding to the image to be labeled based on a set of image labels and image text;

[0127] The second determining unit 708, connected to the first determining unit 706, is used to determine the annotation result of the image to be annotated based on the target label.

[0128] It should be noted that the input unit 702 in this embodiment can be used to execute the above step S202, the acquisition unit 704 in this embodiment can be used to execute the above step S204, the first determination unit 706 in this embodiment can be used to execute the above step S206, and the second determination unit 708 in this embodiment can be used to execute the above step S208.

[0129] The above modules input the images to be labeled into an image classification model and an optical character recognition model. The image classification model is a model trained using a set of sample images. The set of sample images are images obtained from an image library based on target keywords, and the target keywords are keywords determined based on a set of preset tags. The modules then obtain a set of image tags corresponding to the images to be labeled, output by the image classification model, and image text corresponding to the images to be labeled, output by the optical character recognition model. Each set of image tags includes a set of image style tags and a set of image theme tags. The set of image style tags includes primary and secondary image style tags, and the set of image theme tags includes primary and secondary image theme tags. Based on the image tags and image text, the target tags corresponding to the images to be labeled are determined. Based on the target tags, the labeling result for the images to be labeled is determined. This approach addresses the problem of low labeling efficiency in related image labeling methods and improves image labeling efficiency.

[0130] In one exemplary embodiment, the first determining unit includes:

[0131] The first selection module is used to select the image style tag with the highest probability and greater than or equal to a first preset probability threshold from a set of image style tags as the target style tag, and to select the image theme tag with the highest probability and greater than or equal to a second preset probability threshold from a set of image theme tags as the target theme tag, based on the probability corresponding to each image tag in a set of image tags.

[0132] The first determination module is used to determine the image text as the target recognition label when the image text conforms to the preset regular expression rules;

[0133] The target tags include target style tags, target theme tags, and target identification tags.

[0134] In one exemplary embodiment, the above-described apparatus further includes:

[0135] The execution unit is used to select target keywords from a set of preset labels before inputting the image to be labeled into the image classification model and the optical character recognition model, and to obtain a set of candidate images from the image library based on the target keywords. The set of preset labels includes a set of topic labels and a set of style labels, and each image in the candidate image set is associated with the target keyword.

[0136] The third determining unit is used to determine a set of sample images based on the candidate image set;

[0137] The training unit is used to train the image style recognition model and the image subject recognition model to be trained using a set of sample images, respectively, to obtain the image style recognition model and the image subject recognition model. The image classification model to be trained includes the image style recognition model and the image subject recognition model, and the image classification model includes the image style recognition model and the image subject recognition model.

[0138] In one exemplary embodiment, the execution unit includes:

[0139] The second selection module is used to select one preset tag from a set of preset tags as the target keyword; or...

[0140] The first combination module is used to combine at least two preset tags from a set of preset tags into a target keyword; or...

[0141] The second combination module is used to combine at least one preset tag from a set of preset tags with other keywords to form a target keyword, wherein the other keywords are tags of other types besides a set of topic tags and a set of style tags.

[0142] In one exemplary embodiment, the third determining unit includes:

[0143] The filtering module is used to filter a set of candidate images from the candidate image set based on the relevance of each image in the candidate image set to the target keyword, wherein the relevance of each image in the set of candidate images to the target keyword is greater than or equal to a preset threshold.

[0144] The adjustment module is used to adjust the image size of each image in a set of candidate images to a preset size, thereby obtaining a set of sample images.

[0145] In one exemplary embodiment, the training unit includes:

[0146] The first training module is used to train the image style recognition model to be trained using a set of sample images and the corresponding actual image style labels. The training continues until the first target loss value of the image style recognition model to be trained meets the first preset convergence condition, at which point the training ends and the image style recognition model to be trained at the end of the training is determined as the image style recognition model. The first target loss value is the loss value between the image style label determined by the image style recognition model based on the input image to be trained and the actual image style label corresponding to the input image to be trained. The image style recognition model is used to determine the secondary image style label of the image to be labeled and to determine the parent image style label of the secondary image style label of the image to be labeled as the primary image style label of the image to be labeled.

[0147] The second training module is used to train the image topic recognition model to be trained using a set of sample images and the corresponding actual image topic labels. Training continues until the second target loss value of the image topic recognition model meets the second preset convergence condition, at which point the training ends, and the image topic recognition model at the end of training is determined as the image topic recognition model. The second target loss value is the loss value between the image topic labels determined by the image topic recognition model based on the input image to be trained and the actual image topic labels corresponding to the input image to be trained. The image topic recognition model is used to determine the secondary image topic labels of the image to be labeled, and the parent image topic labels of the secondary image topic labels of the image to be labeled are determined as the primary image topic labels of the image to be labeled.

[0148] In one exemplary embodiment, the training unit further includes:

[0149] The third training module is used to train the first image style recognition model using a set of sample images and the actual first-level image style labels corresponding to the set of sample images, so as to obtain the first image style recognition model.

[0150] The fourth training module is used to train the second image style recognition model using a set of sample images and the actual secondary image style labels corresponding to the set of sample images, so as to obtain the second image style recognition model.

[0151] The fifth training module is used to train the first image topic recognition model based on a set of sample images and the actual first-level image topic labels corresponding to the set of sample images, so as to obtain the first image topic recognition model.

[0152] The sixth training module is used to train the second image topic recognition model based on a set of sample images and the actual secondary image topic labels corresponding to the set of sample images, so as to obtain the second image topic recognition model.

[0153] The image style recognition model includes a first image style recognition model and a second image style recognition model, and the image theme recognition model includes a first image theme recognition model and a second image theme recognition model. The first image style recognition model is used to determine the primary image style label of the image to be labeled, and the second image style recognition model is used to determine the secondary image style label of the image to be labeled. The first image theme recognition model is used to determine the primary image theme label of the image to be labeled, and the second image theme recognition model is used to determine the secondary image theme label of the image to be labeled.

[0154] In one exemplary embodiment, the second determining unit includes:

[0155] The second determining module is used to determine the probability of a target label corresponding to another target label as the labeling result for the image to be labeled; or,

[0156] The third determination module is used to determine the target label as the annotation result for the image to be annotated.

[0157] It should be noted that the examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the content disclosed in the above embodiments. It should also be noted that the above modules, as part of a device, can operate in environments such as... Figure 1 The hardware environment shown can be implemented through software or hardware, and the hardware environment includes the network environment.

[0158] According to another aspect of the embodiments of this application, a storage medium is also provided. Optionally, in this embodiment, the storage medium can be used to execute the program code of any of the image annotation methods described above in the embodiments of this application.

[0159] Optionally, in this embodiment, the storage medium may be located on at least one of the network devices in the network shown in the above embodiment.

[0160] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps:

[0161] S1, Input the image to be labeled into the image classification model and the optical character recognition model. The image classification model is the model obtained by training the image classification model with a set of sample images. The set of sample images are images obtained from the image library according to the target keywords. The target keywords are keywords determined according to a set of preset labels.

[0162] S2, obtain a set of image labels corresponding to the image to be labeled, output by the image classification model, and obtain the image text corresponding to the image to be labeled, output by the optical character recognition model. The set of image labels includes a set of image style labels and a set of image theme labels. The set of image style labels includes a first-level image style label and a second-level image style label. The set of image theme labels includes a first-level image theme label and a second-level image theme label.

[0163] S3, based on a set of image labels and image text, determine the target label corresponding to the image to be labeled;

[0164] S4. Based on the target label, determine the annotation results for the image to be annotated.

[0165] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments, and will not be repeated in this embodiment.

[0166] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, ROMs, RAMs, portable hard drives, magnetic disks, or optical disks.

[0167] According to another aspect of the embodiments of this application, an electronic device for implementing the above-described image annotation method is also provided. The electronic device may be a server, a terminal, or a combination thereof.

[0168] Figure 8 This is a structural block diagram of an optional electronic device according to an embodiment of this application, such as... Figure 8 As shown, it includes a processor 802, a communication interface 804, a memory 806, and a communication bus 808. The processor 802, communication interface 804, and memory 806 communicate with each other via the communication bus 808.

[0169] Memory 806 is used to store computer programs;

[0170] When processor 802 executes a computer program stored in memory 806, it performs the following steps:

[0171] S1, Input the image to be labeled into the image classification model and the optical character recognition model. The image classification model is the model obtained by training the image classification model with a set of sample images. The set of sample images are images obtained from the image library according to the target keywords. The target keywords are keywords determined according to a set of preset labels.

[0172] S2, obtain a set of image labels corresponding to the image to be labeled, output by the image classification model, and obtain the image text corresponding to the image to be labeled, output by the optical character recognition model. The set of image labels includes a set of image style labels and a set of image theme labels. The set of image style labels includes a first-level image style label and a second-level image style label. The set of image theme labels includes a first-level image theme label and a second-level image theme label.

[0173] S3, based on a set of image labels and image text, determine the target label corresponding to the image to be labeled;

[0174] S4. Based on the target label, determine the annotation results for the image to be annotated.

[0175] Optionally, in this embodiment, the communication bus can be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, Figure 8 The symbol is represented by a single thick line, but this does not indicate that there is only one bus or one type of bus. The communication interface is used for communication between the aforementioned electronic device and other devices.

[0176] The aforementioned memory may include RAM, or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0177] As an example, the memory 806 described above may include, but is not limited to, the input unit 702, the acquisition unit 704, the first determination unit 706, and the second determination unit 708 from the image annotation device described above. Furthermore, it may include, but is not limited to, other module units from the image annotation device described above, which will not be elaborated upon in this example.

[0178] The processors mentioned above can be general-purpose processors, including but not limited to: CPU (Central Processing Unit), NP (Network Processor), etc.; they can also be DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0179] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments, and will not be repeated here.

[0180] Those skilled in the art will understand that Figure 8 The structure shown is for illustrative purposes only. The device implementing the above image annotation method can be a terminal device, such as a smartphone (e.g., Android phone, iOS phone), tablet computer, PDA, mobile Internet Device (MID), PAD, etc. Figure 8 This does not limit the structure of the aforementioned electronic device. For example, the electronic device may also include components that are more... Figure 8 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 8 The different configurations shown.

[0181] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, ROM, RAM, disk or optical disk, etc.

[0182] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0183] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0184] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0185] In the several embodiments provided in this application, it should be understood that the disclosed client can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between units or modules, and may be electrical or other forms.

[0186] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the solution provided in this embodiment, depending on actual needs.

[0187] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0188] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for annotating images, characterized in that, include: Select target keywords from a set of preset tags, and obtain a set of candidate images from an image library based on the target keywords. The set of preset tags includes a set of theme tags and a set of style tags. Each image in the set of candidate images is associated with the target keyword. Based on the set of candidate images, determine a set of sample images. Use the set of sample images to train the image style recognition model and the image theme recognition model to obtain the image style recognition model and the image theme recognition model, respectively. The images to be labeled are input into an image classification model and an optical character recognition model. The image classification model is a model obtained by training the image classification model to be trained using the set of sample images. The set of sample images are images obtained from the image library according to the target keywords. The target keywords are keywords determined according to the set of preset labels. The image classification model to be trained includes the image style recognition model and the image topic recognition model. Obtain a set of image tags output by the image classification model corresponding to the image to be labeled, and obtain the image text output by the optical character recognition model corresponding to the image to be labeled. The set of image tags includes a set of image style tags and a set of image theme tags. The set of image style tags includes a first-level image style tag and a second-level image style tag. The set of image theme tags includes a first-level image theme tag and a second-level image theme tag. Determining the target label corresponding to the image to be labeled based on the set of image labels and the image text includes: selecting the image style label with the highest probability and greater than or equal to a first preset probability threshold from the set of image style labels as the target style label, and selecting the image theme label with the highest probability and greater than or equal to a second preset probability threshold from the set of image theme labels as the target theme label; and determining the image text as the target recognition label if the image text conforms to a preset regular expression rule; wherein, the target label includes the target style label, the target theme label, and the target recognition label; Based on the target label, determine the annotation result for the image to be annotated.

2. The method according to claim 1, characterized in that, The step of selecting the target keyword from the set of preset tags includes: Select one preset tag from the set of preset tags as the target keyword; or... Combine at least two preset tags from the set of preset tags to form the target keyword; or... At least one preset tag from the set of preset tags is combined with other keywords to form the target keyword, wherein the other keywords are tags of other types besides a set of topic tags and a set of style tags.

3. The method according to claim 1, characterized in that, The step of determining the set of sample images based on the candidate image set includes: Based on the correlation between each image in the candidate image set and the target keyword, a set of candidate images is selected from the candidate image set, wherein the correlation between each image in the set of candidate images and the target keyword is greater than or equal to a preset threshold. The image size of each image in the set of candidate images is adjusted to a preset size to obtain the set of sample images.

4. The method according to claim 1, characterized in that, The process involves using the set of sample images to train both the image style recognition model and the image subject recognition model, resulting in the image style recognition model and the image subject recognition model, respectively. The image style recognition model to be trained is trained using the set of sample images and the corresponding actual image style labels. Training ends when the first target loss value of the image style recognition model to be trained meets the first preset convergence condition, and the image style recognition model to be trained at the end of training is determined as the image style recognition model. The first target loss value is the loss value between the image style label determined by the image style recognition model based on the input image to be trained and the actual image style label corresponding to the input image to be trained. The image style recognition model is used to determine the secondary image style label of the image to be labeled and to determine the parent image style label of the secondary image style label of the image to be labeled as the primary image style label of the image to be labeled. The image topic recognition model is trained using the set of sample images and the corresponding actual image topic labels. Training continues until the second target loss value of the image topic recognition model meets the second preset convergence condition. The training ends, and the image topic recognition model at the end of training is determined as the image topic recognition model. The second target loss value is the loss value between the image topic labels determined by the image topic recognition model based on the input image and the actual image topic labels corresponding to the input image. The image topic recognition model is used to determine the secondary image topic labels of the image to be labeled and to determine the parent image topic labels of the secondary image topic labels of the image to be labeled as the primary image topic labels of the image to be labeled.

5. The method according to claim 1, characterized in that, The step of training the image style recognition model and the image subject recognition model using the set of sample images to obtain the image style recognition model and the image subject recognition model respectively, further includes: Using the set of sample images and the actual first-level image style labels corresponding to the set of sample images, the first image style recognition model to be trained is trained to obtain the first image style recognition model. Using the set of sample images and the actual secondary image style labels corresponding to the set of sample images, the second image style recognition model is trained to obtain the second image style recognition model. Based on the set of sample images and the actual first-level image topic labels corresponding to the set of sample images, the first image topic recognition model is trained to obtain the first image topic recognition model. Based on the set of sample images and the actual secondary image topic labels corresponding to the set of sample images, the second image topic recognition model is trained to obtain the second image topic recognition model. The image style recognition model includes a first image style recognition model and a second image style recognition model, and the image theme recognition model includes a first image theme recognition model and a second image theme recognition model. The first image style recognition model is used to determine the primary image style tag of the image to be labeled, and the second image style recognition model is used to determine the secondary image style tag of the image to be labeled. The first image theme recognition model is used to determine the primary image theme tag of the image to be labeled, and the second image theme recognition model is used to determine the secondary image theme tag of the image to be labeled.

6. The method according to claim 1, characterized in that, The step of determining the annotation result for the image to be annotated based on the target label includes: The probability of the target label corresponding to the target label is determined as the annotation result for the image to be annotated; or... The target label is determined as the annotation result of the image to be annotated.

7. An image annotation device, characterized in that, The apparatus is used to implement the method of any one of claims 1 to 6, comprising: An execution unit is configured to select target keywords from a set of preset tags and obtain a set of candidate images from an image library based on the target keywords. The set of preset tags includes a set of theme tags and a set of style tags, and each image in the candidate image set is associated with the target keyword. A third determination unit is configured to determine a set of sample images based on the candidate image set. A training unit is configured to use the set of sample images to train the image style recognition model and the image theme recognition model to be trained, respectively, to obtain the image style recognition model and the image theme recognition model. An input unit is used to input the image to be labeled into an image classification model and an optical character recognition model. The image classification model is a model obtained by training the image classification model to be trained using the set of sample images. The set of sample images are images obtained from the image library according to the target keywords. The target keywords are keywords determined according to the set of preset labels. The image classification model to be trained includes the image style recognition model to be trained and the image topic recognition model to be trained. The image classification model includes the image style recognition model and the image topic recognition model. The acquisition unit is used to acquire a set of image tags output by the image classification model corresponding to the image to be labeled, and to acquire image text output by the optical character recognition model corresponding to the image to be labeled. The set of image tags includes a set of image style tags and a set of image theme tags. The set of image style tags includes a first-level image style tag and a second-level image style tag. The set of image theme tags includes a first-level image theme tag and a second-level image theme tag. The first determining unit is configured to determine a target label corresponding to the image to be labeled based on the set of image labels and the image text, comprising: a first selecting module, configured to select, based on the probability corresponding to each image label in the set of image labels, the image style label with the highest probability and greater than or equal to a first preset probability threshold as the target style label, and the image theme label with the highest probability and greater than or equal to a second preset probability threshold as the target theme label from the set of image theme labels; and a first determining module, configured to determine the image text as a target recognition label if the image text conforms to a preset regular expression rule; wherein, the target label includes the target style label, the target theme label, and the target recognition label; The second determining unit is used to determine the annotation result of the image to be annotated based on the target label.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program, when executed, performs the method of any one of claims 1 to 6.

9. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the method of any one of claims 1 to 6 through the computer program.

Images