Label data processing method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a label data processing method, which comprises: obtaining label data of a to-be-processed picture by a multi-modal large model; based on the label data, performing matching processing in a preset label library to obtain a matching result corresponding to the label data, the preset label library comprising labels of multiple levels; and determining a target label of the to-be-processed picture based on the level of the matching result. By obtaining the label data of the to-be-processed picture by the multi-modal large model, since the preset label library comprises labels of multiple levels, after the matching processing in the preset label library based on the label data, the matching result related to different levels can be obtained, the target label can be determined according to the matching result related to different levels, the occurrence of multiple variants can be reduced, and thus the recognition accuracy of the learning machine in actual application is improved.

Description

Technical Field

[0001] This invention relates to the field of data processing, and more particularly to a tag data processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the increasing application of multimodal large modeling, such as in learning machines for image recognition, traditional image annotation methods are often used. This involves directly providing the user with the multimodal large model's output of the photograph, such as "table," "white table," "wooden table," and "IKEA table." The diversity and inconsistency of these outputs can interfere with establishing a unified standard for recognizing common item names, leading to various variations in the recognition results and consequently reducing accuracy. Therefore, providing a label data processing method that improves recognition accuracy has become a pressing issue. Summary of the Invention

[0003] This invention provides a label data processing method aimed at addressing the low recognition accuracy of existing label data processing methods for devices such as image processing machines. By acquiring label data of the image to be processed using a multimodal large model, and considering that the preset label library includes labels at multiple levels, matching processing based on the label data within the preset label library yields matching results related to different levels. Based on these matching results, the target label can be determined, reducing the occurrence of various variations and thus improving the recognition accuracy of image processing machines in practical applications.

[0004] In a first aspect, embodiments of the present invention provide a tag data processing method, the method comprising the following steps:

[0005] Obtain the label data of the image to be processed from a multimodal large model;

[0006] Based on the tag data, a matching process is performed in a preset tag library to obtain the matching result corresponding to the tag data. The preset tag library includes tags at multiple levels.

[0007] Based on the hierarchy of the matching results, the target label of the image to be processed is determined.

[0008] Optionally, before obtaining the label data of the multimodal large model for the image to be processed, the method further includes:

[0009] Retrieve multiple preset tags;

[0010] Among the multiple preset tags, multiple parent tags are determined;

[0011] Each parent label is used as a classification target. Among the multiple preset labels, the child label corresponding to each parent label is determined, and the parent label has a higher level than the child label.

[0012] Based on the parent tag and the child tags corresponding to the parent tag, the preset tag library is constructed.

[0013] Optionally, the matching result includes at least one matching tag, each matching tag including a matching confidence level with the tag data, and determining the target tag of the image to be processed based on the hierarchy of the matching result includes:

[0014] Based on the matching confidence, the target matching label is selected from the matching results;

[0015] Based on the hierarchy of the target matching tags, the target tags of the image to be processed are determined.

[0016] Optionally, the hierarchy of the target matching tag is a parent tag or a child tag, and determining the target tag of the image to be processed based on the hierarchy of the target matching tag includes:

[0017] When the target matching tag is at the level of the sub-tag, the target matching tag is used as the target tag of the image to be processed;

[0018] When the target matching tag is at the level of the parent tag, matching processing is performed in a preset image library based on the target matching tag to obtain a set of sub-tag images, and the target tag of the image to be processed is determined based on the set of sub-tag images.

[0019] Optionally, the preset image library includes at least one image, and each image includes an image tag. The matching process based on the target matching tag in the preset image library to obtain a sub-tag image set includes:

[0020] Calculate the correlation between the target matching tag and each of the image tags;

[0021] Based on the relevance, select the sub-tag image corresponding to the target matching tag from the preset image library;

[0022] Based on the sub-label images, the sub-label image set is constructed.

[0023] Optionally, determining the target tag of the image to be processed based on the sub-tag image set includes:

[0024] Based on a preset image search engine, the target image is determined from the sub-tag image set;

[0025] The tags of the target image are used as the target tags of the image to be processed.

[0026] Optionally, the step of determining the target image from the sub-tag image set based on a preset image search engine includes:

[0027] The sub-tag image set and the image to be processed are provided to the preset image search engine so that the image search engine can calculate the image similarity between each sub-tag image in the sub-tag image set and the image to be processed.

[0028] Based on the image similarity, the target image is determined from the sub-label image set.

[0029] Secondly, embodiments of the present invention also provide a tag data processing apparatus, the tag data processing apparatus comprising:

[0030] The first acquisition module is used to acquire the label data of the multimodal large model for the image to be processed;

[0031] The first matching module is used to perform matching processing in a preset tag library based on the tag data to obtain the matching result corresponding to the tag data. The preset tag library includes tags at multiple levels.

[0032] The first determining module is used to determine the target label of the image to be processed based on the hierarchy of the matching results.

[0033] Thirdly, embodiments of the present invention provide an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps in the tag data processing method provided in embodiments of the present invention.

[0034] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in the tag data processing method provided in the embodiments of the present invention.

[0035] In this embodiment of the invention, a multimodal large model is used to acquire label data of the image to be processed; based on the label data, matching processing is performed in a preset label library to obtain matching results corresponding to the label data. The preset label library includes labels at multiple levels; based on the level of the matching results, the target label of the image to be processed is determined. By acquiring the label data of the image to be processed using a multimodal large model, since the preset label library includes labels at multiple levels, matching processing based on the label data in the preset label library can obtain matching results related to different levels. Based on the matching results related to different levels, the target label can be determined, reducing the occurrence of multiple variations and thus improving the recognition accuracy of the image recognition machine in practical applications. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 This is a flowchart of a tag data processing method provided in an embodiment of the present invention;

[0038] Figure 2 This is a flowchart of another tag data processing method provided in an embodiment of the present invention;

[0039] Figure 3 This is a schematic diagram of the structure of a tag data processing device provided in an embodiment of the present invention;

[0040] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] like Figure 1 As shown, Figure 1 This is a flowchart of a tag data processing method provided in an embodiment of the present invention, including:

[0043] 101. Obtain the label data of the image to be processed from the multimodal large model.

[0044] In this embodiment of the invention, the above-described label data processing method can be applied to a label data management platform (or can be understood as a label data management system). The label data management platform can be constructed from a server or server cluster, and the server or server cluster can be any electronic device with functions such as data analysis, data processing, data transmission, and data storage. The multimodal large model can be deployed on the label data management platform, and upon obtaining an image to be processed, it can provide the image to the multimodal large model so that the multimodal large model can perform annotation processing on the image to obtain the label data.

[0045] Alternatively, the aforementioned tag data management platform can also deploy an API interface for a multimodal large model. The tag data management platform can call the aforementioned multimodal large model through the aforementioned API interface to perform annotation processing on the aforementioned images to be processed, thereby obtaining the aforementioned tag data.

[0046] The images to be processed are obtained by the aforementioned tag data management platform through the aforementioned data transmission function, or by the user through the graphical user interface (i.e., the graphical user interface of the learning device) or API interface. Specifically, the user can take the images to be processed using the learning device and upload them to the aforementioned tag data management platform through the graphical user interface or the aforementioned API interface. The tag data management platform then implements the aforementioned tag data processing method to obtain the target tag and feeds the target tag back to the learning device for the user to learn from.

[0047] The aforementioned multimodal large model can be any artificial intelligence model capable of processing and understanding multiple types of input (such as text, images, and sound). It can integrate information from different sources to provide more comprehensive analysis and responses. For example, it can comprehensively analyze the visual content, contextual information, and other modal features of an image to identify key elements and concepts within it. Specifically, the aforementioned multimodal large model can be any of the following: chatgpt large model, CLIP large model, DALL·E large model, MMM (Google's Multimodal Mixture Model), or other multimodal large models.

[0048] Before outputting the image to be processed to the multimodal large model, the image can be preprocessed to ensure that it meets the input requirements of the multimodal large model. This preprocessing may include resizing, pixel value normalization, data augmentation, and noise reduction.

[0049] After the above multimodal large model annotates the images to be processed, it can output a set of label data describing the content of the images, i.e., A = [(label max type max δ max ], where the above label max This represents a label or category for a specific element in an image. For example, if it's an object, it could be "cat," "table," or "car"; if it's a scene, it could be "forest," "city," etc. The above type... max This is represented by the specific type or attribute of the element. For example, if it's an animal, it could be "breed" (e.g., Maine Coon, Siberian tiger); if it's furniture, it could be "material" (e.g., wooden table, marble table); if it's a vehicle, it could be "vehicle type" (e.g., SUV, etc.). max This represents the confidence level of matching the image content, used to measure the accuracy of the label. For example, if the probability of an element being identified as "dog" is 90%, the corresponding δ is... max =0.9.

[0050] The aforementioned label data is generated through the internal algorithm of a multimodal large model, reflecting the main features and related information of the image. The output of the multimodal model (i.e., the label data) must conform to the following formula:

[0051] f(label i )=max(δ i )

[0052] The above label i The δ is represented by the label or category of the i-th element in the image. i Max(δ) represents the confidence score of the label or category of the i-th element. i f(label) represents selecting the largest value from multiple confidence values ​​as the final confidence assessment for that label or category. i ) represents the final confidence result for that label or category, i.e., δ i The maximum value.

[0053] 102. Based on the tag data, perform matching processing in the preset tag library to obtain the matching results corresponding to the tag data.

[0054] In this embodiment of the invention, the preset tag library includes tags at multiple levels. The tags at multiple levels may include parent tags and child tags. The parent tag may be at a higher level than the child tags. For example, "table" may be the parent tag, and the child tags corresponding to "table" may be "octagonal table, white table, wooden table, dining table, etc."

[0055] The matching process described above can be exact matching, fuzzy matching, or semantic matching. Exact matching can be a direct search in a preset tag library to see if there are any completely identical tags. Fuzzy matching allows for a certain degree of tag difference (e.g., spelling errors can be allowed) during the search in the preset tag library. Semantic matching can be based on the semantic relevance of tags in the tag data.

[0056] Specifically, the matching results may include at least one matching tag. The type (i.e., specific type) in the tag data can be used as a classification criterion, and tags related to the type can be classified in the preset tag library as matching tags.

[0057] 103. Based on the hierarchy of the matching results, determine the target label of the image to be processed.

[0058] In this embodiment of the invention, the matching results may include at least one matching tag, and each matching tag includes a matching confidence level with the tag data. Based on the matching confidence level, the matching tag with the highest matching confidence level (i.e., the target matching tag) can be selected from the matching results. The target tag for the image to be processed is determined based on the hierarchy of the matching tags.

[0059] It should be noted that, in order to reduce the diversity and inconsistency of the output results and to avoid multiple variations in the recognition results, the target label needs to be a relatively specific label (i.e., a sub-label), rather than a vague label (i.e., a parent label). Therefore, when the target matching label is at the sub-label level, it can be directly used as the target label. Conversely, when the target matching label is at the parent label level, it cannot be directly used as the target label. In this case, it is necessary to determine the sub-labels related to the parent label based on the parent label, and then determine one sub-label from the related sub-labels as the target label.

[0060] Understandably, since the target matching tags are obtained from a preset tag library, which can be categorized according to common items, when the target matching tags are sub-tags and used as target tags, the target tags can achieve a relatively standardized level (i.e., in the image to be recognized, if the item type remains unchanged, even if other images are used, the output target tags will be the same). This can improve the accuracy of the image recognition results of the learning machine, enabling it to serve users more reliably, improve the user's operating experience, and increase user trust and satisfaction with the learning machine.

[0061] In this embodiment of the invention, a multimodal large model is used to acquire label data of the image to be processed; based on the label data, matching processing is performed in a preset label library to obtain matching results corresponding to the label data. The preset label library includes labels at multiple levels; based on the level of the matching results, the target label of the image to be processed is determined. By acquiring the label data of the image to be processed using a multimodal large model, since the preset label library includes labels at multiple levels, matching processing based on the label data in the preset label library can obtain matching results related to different levels. Based on the matching results related to different levels, the target label can be determined, reducing the occurrence of multiple variations and thus improving the recognition accuracy of the image recognition machine in practical applications.

[0062] It is understood that in the specific implementation of this application, data such as images to be processed, tag data, and sub-tag image sets are involved. When the embodiments in this application are applied to specific products or technologies, user permission or consent is required. Furthermore, the collection, use and processing of related data, as well as the construction, training and use of tools such as multimodal large models, tag libraries, and image libraries, must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0063] It should be noted that the tag data processing method provided in this embodiment of the invention can be applied to devices such as computers and servers that can process tag data.

[0064] Optionally, before obtaining the label data of the multimodal large model for the image to be processed, multiple preset labels can be obtained; multiple parent labels are determined among the multiple preset labels; each parent label is used as a classification target, and child labels corresponding to each parent label are determined among the multiple preset labels, with the parent label having a higher level than the child label; a preset label library is constructed based on the parent labels and the child labels corresponding to the parent labels.

[0065] In this embodiment of the invention, the aforementioned preset labels can be obtained through the data transmission function by acquiring publicly available datasets, open-source projects, industry standard directories, or other similar channels. After obtaining the preset labels, a hierarchical structure of the labels can be constructed and maintained using classification tools such as Excel or tools like pandas and matplotlib in the Python programming language. This hierarchical structure can be a tree structure or a graph.

[0066] Alternatively, the aforementioned preset labels and classification prompts can be directly provided to the multimodal large-scale model, enabling it to classify the preset labels, obtain at least one parent label, and use each parent label as a classification target. From the multiple preset labels, the model then determines the child labels corresponding to each parent label. The classification prompts could be something like, "Please determine the parent label from the following labels and classify the child labels corresponding to each parent label, where the parent label's hierarchy must be higher than the child labels."

[0067] Alternatively, the above programming language can be used to directly construct a classification project, and the preset labels can be input into the classification project for classification processing to obtain the parent label and the child label corresponding to each parent label.

[0068] It should be noted that each parent tag can correspond to multiple child tags, but there may also be cases where there is only a parent tag without corresponding child tags. The above classification process must follow the general rules of item classification. That is, an object may generally be called corn, but in a specific region describing a specific use, it may also be called maize. In this case, the object must follow the general rules of item classification, and its tag must be determined as corn.

[0069] For example, assuming the preset labels include [table, fruit, clothes, octagonal table, white table, wooden table, dining table, apple, pineapple, peach, banana, short-sleeved shirt, sweater, shorts, beach shorts], then the preset labels can be divided into the following sets according to the general classification rules of items using any of the above classification methods:

[0070] Parent tags include:

[0071] Set P1[(table, type=0)]; Set P2[(fruit, type=0)]; Set P3[(clothes, type=0)].

[0072] In this context, if type = 0 in the set, it indicates that the tag's hierarchical relationship is that of a parent tag; if type = 1 in the set, it indicates that the tag's hierarchical relationship is that of a child tag.

[0073] The sub-tag sets are as follows:

[0074] S1 = [(Octagonal table, type=1), (White table, type=1), (Wooden table, type=1), (Dining table, type=1)]

[0075] S2 = [(Apple, type=1), (Pineapple, type=1), (Peach, type=1), (Banana, type=1)]

[0076] S3 = [(Short-sleeved shirt, type=1), (Sweater, type=1), (Shorts, type=1), (Beach shorts, type=1)]

[0077] From the above division results, we can see that: That is, the set of sub-labels S1 belongs to the set P1, the set of sub-labels S2 belongs to the set P2, and the set of sub-labels S3 belongs to the set P3.

[0078] Optionally, the matching results include at least one matching label, and each matching label includes a matching confidence level with the label data. In the step of determining the target label of the image to be processed based on the matching results, the target matching label can also be selected from the matching results based on the matching confidence level; and the target label of the image to be processed can be determined based on the level of the target matching label.

[0079] In this embodiment of the invention, the matching process described above can be semantic matching, which can be implemented based on vector space models, machine learning models, etc.

[0080] For example, word embedding techniques such as Word2Vec, GloVe, or BERT can be used to convert tags into vector form. In the vector space, semantically similar tags are mapped to close positions. Therefore, the distance between these vectors (e.g., cosine similarity) can be calculated to quantify the semantic relevance between tags, and the matching results can be determined based on the semantic relevance (e.g., the top 100, top 50, or top 10 tags with the highest semantic relevance to the labeled data can be extracted from the tag library as matching tags and added to the matching results).

[0081] Alternatively, a machine learning model can be trained to identify the semantic relationships between each tag and the tag data in a pre-defined tag library. Specifically, supervised learning can be used, and the machine learning model could be a support vector machine (SVM) or a more complex neural network.

[0082] As can be seen from the above matching process, when the vector space model and the machine learning model output the matching results, they will simultaneously output the matching confidence level corresponding to each matching label. For example, the higher the semantic relevance between the matching label and the label data, the higher the matching confidence level can be, and vice versa.

[0083] Therefore, based on the matching confidence level mentioned above, the matching tag with the highest matching confidence level can be selected as the target matching tag from the matching results. When the hierarchical relationship of the target matching tag is a sub-tag, the target matching tag can be used as the target tag mentioned above.

[0084] In one possible embodiment, when the hierarchical relationship of the target matching label is a parent label, the target matching label can be removed, and new target matching labels can be reselected from the matching results in descending order of matching confidence, until the hierarchical relationship of the new target matching label is a child label, and then the new target matching label is used as the target label.

[0085] Optionally, the target matching tag can be a parent tag or a child tag. In the step of determining the target tag of the image to be processed based on the level of the target matching tag, when the level of the target matching tag is a child tag, the target matching tag can be used as the target tag of the image to be processed; when the level of the target matching tag is a parent tag, matching processing is performed in a preset image library based on the target matching tag to obtain a set of child tag images, and the target tag of the image to be processed is determined based on the set of child tag images.

[0086] In this embodiment of the invention, when the target matching tag is a sub-tag, it means that there will be no problem of multiple variations when the above sub-tag is used as the target tag. Therefore, the above target matching tag can be directly used as the above target tag.

[0087] When the target matching tag is the parent tag, there are usually many child tags corresponding to a parent tag. Therefore, if the parent tag is used as the target tag, there is a high probability of multiple variations. Therefore, the target matching tag can be used to perform matching processing in a preset image library to obtain a set of child tag images. Then, an image can be selected from the set of child tag images as the target image, and the tag of the target image can be used as the target tag.

[0088] Optionally, the preset image library includes at least one image, and each image includes an image tag. In the step of performing matching processing in the preset image library based on the target matching tag to obtain a sub-tag image set, the relevance between the target matching tag and each image tag can also be calculated; based on the relevance, the sub-tag images corresponding to the target matching tags are selected in the preset image library; and based on the sub-tag images, a sub-tag image set is constructed.

[0089] In this embodiment of the invention, the matching process described above may involve identifying images that are similar to or related to the labeled data at the label level (i.e., the text level) and using them as sub-label images, thereby constructing the sub-label image set based on the sub-label images.

[0090] Specifically, the aforementioned relevance can be understood as the aforementioned semantic relevance. The matching process in the aforementioned preset image library can also be semantic matching, and the aforementioned semantic matching can also be implemented based on the aforementioned vector space model, the aforementioned machine learning model, etc.

[0091] For example, assuming the target matching tag is "fruit", images with sub-tags related to fruit can be matched from the preset image library. These images can include images related to mango, dragon fruit, orange, etc. Each of these images is treated as a sub-tag image and added to a preset empty set. After adding the images, the set of sub-tag images is obtained.

[0092] Optionally, in the step of determining the target label of the image to be processed based on the sub-label image set, the target image can also be determined from the sub-label image set based on a preset image search engine; and the label of the target image can be used as the target label of the image to be processed.

[0093] In this embodiment of the invention, the image search engine can be understood as an image-based search engine. By simultaneously providing the image to be processed and the set of sub-tag images as input, the target image can be determined. Furthermore, the tags of the target image can be used as the target tags of the image to be processed.

[0094] Optionally, in the step of determining the target image from the sub-label image set based on a preset image search engine, the sub-label image set and the image to be processed can also be provided to the preset image search engine so that the image search engine can calculate the image similarity between each sub-label image in the sub-label image set and the image to be processed; and the target image can be determined from the sub-label image set based on the image similarity.

[0095] In this embodiment of the invention, the above-mentioned sub-tag image set and the above-mentioned image to be processed can be provided as input to the above-mentioned image search engine. The above-mentioned image search engine can calculate the image similarity between each sub-tag image in the sub-tag image set and the image to be processed, and based on the above-mentioned image similarity, determine the sub-tag image with the highest image similarity in the sub-tag image set as the above-mentioned target image.

[0096] like Figure 2 As shown in the figure, this embodiment of the invention also provides a flowchart of another label data processing method, including the following steps:

[0097] Step 1: Input the image P (i.e. the image to be identified) into the multimodal large model for annotation processing to obtain the output label [A] (i.e. label data);

[0098] Step 2: Use the label determiner to determine whether the label data is a child label or a parent label. When the label data is a child label, the child label with the highest confidence can be selected as the target label for output.

[0099] Step 3: When the specific type of the label data is the parent label, filtering (or matching) can be performed in the image set to determine the child label image set [I];

[0100] Step 4: Provide the sub-label image set [I] and the original image P (i.e. the image to be processed) to the image search engine so that the image search engine can identify the image with the highest image similarity to the image to be processed in the sub-label image set as the target image, and output the label of the target image as the target label.

[0101] like Figure 3 As shown, this embodiment of the invention also provides a tag data processing device, including:

[0102] The first acquisition module 301 is used to acquire the label data of the multimodal large model of the image to be processed;

[0103] The first matching module 302 is used to perform matching processing in a preset tag library based on the tag data to obtain the matching result corresponding to the tag data. The preset tag library includes tags at multiple levels.

[0104] The first determining module 303 is used to determine the target label of the image to be processed based on the level of the matching result.

[0105] Optionally, the tag data processing device further includes:

[0106] The second acquisition module is used to acquire multiple preset tags;

[0107] The second determining module is used to determine multiple parent tags among the multiple preset tags;

[0108] The third determining module is used to take each parent label as a classification target and determine the child label corresponding to each parent label among a plurality of preset labels, wherein the level of the parent label is higher than the level of the child label;

[0109] The first construction module is used to construct the preset tag library based on the parent tag and the child tags corresponding to the parent tag.

[0110] Optionally, the matching result includes at least one matching tag, and each matching tag includes a matching confidence level with the tag data. The first determining module 303 includes:

[0111] The first selection submodule is used to select a target matching label from the matching results based on the matching confidence.

[0112] The first determining submodule is used to determine the target label of the image to be processed based on the hierarchy of the target matching label.

[0113] Optionally, the target matching tag is hierarchically defined as a parent tag or a child tag, and the first determining submodule is further configured to:

[0114] When the target matching tag is at the level of the sub-tag, the target matching tag is used as the target tag of the image to be processed;

[0115] When the target matching tag is at the level of the parent tag, matching processing is performed in a preset image library based on the target matching tag to obtain a set of sub-tag images, and the target tag of the image to be processed is determined based on the set of sub-tag images.

[0116] Optionally, the preset image library includes at least one image, and each image includes an image tag. The first determining submodule includes:

[0117] The first calculation unit is used to calculate the correlation between the target matching tag and each of the image tags;

[0118] The first selection unit is used to select, based on the relevance, the sub-tag image corresponding to the target matching tag from the preset image library;

[0119] The first construction unit is used to construct the sub-tag image set based on the sub-tag images.

[0120] Optionally, the first determining submodule further includes:

[0121] The first determining unit is used to determine the target image in the sub-tag image set based on a preset image search engine;

[0122] The first processing unit is used to use the tags of the target image as the target tags of the image to be processed.

[0123] Optionally, the first determining unit includes:

[0124] The first calculation subunit is used to provide the sub-tag image set and the image to be processed to the preset image search engine, so that the image search engine can calculate the image similarity between each sub-tag image in the sub-tag image set and the image to be processed.

[0125] The first determining subunit is used to determine the target image in the sub-label image set based on the image similarity.

[0126] like Figure 4 As shown, this embodiment of the invention also provides an electronic device, characterized in that it includes a processor, which can execute any of the above-described tag data processing methods.

[0127] Specifically, it includes a processor 401 and a memory 402, as well as a computer program stored in the memory 402 and capable of running on the processor 401 to execute the tag data processing method, wherein:

[0128] The processor 401 executes the calculator program for the tag data processing method stored in the memory 402, and performs the following steps:

[0129] Obtain the label data of the image to be processed from a multimodal large model;

[0130] Based on the tag data, a matching process is performed in a preset tag library to obtain the matching result corresponding to the tag data. The preset tag library includes tags at multiple levels.

[0131] Based on the hierarchy of the matching results, the target label of the image to be processed is determined.

[0132] Optionally, before acquiring the label data of the multimodal large model for the image to be processed, the method executed by the processor 401 further includes:

[0133] Retrieve multiple preset tags;

[0134] Among the multiple preset tags, multiple parent tags are determined;

[0135] Each parent label is used as a classification target. Among the multiple preset labels, the child label corresponding to each parent label is determined, and the parent label has a higher level than the child label.

[0136] Based on the parent tag and the child tags corresponding to the parent tag, the preset tag library is constructed.

[0137] Optionally, the matching result includes at least one matching tag, each matching tag including a matching confidence level with the tag data, and the processor 401 performs the hierarchical determination of the target tag of the image to be processed based on the matching result, including:

[0138] Based on the matching confidence, the target matching label is selected from the matching results;

[0139] Based on the hierarchy of the target matching tags, the target tags of the image to be processed are determined.

[0140] Optionally, the hierarchy of the target matching tag is a parent tag or a child tag, and the process 401 executes the step of determining the target tag of the image to be processed based on the hierarchy of the target matching tag, including:

[0141] When the target matching tag is at the level of the sub-tag, the target matching tag is used as the target tag of the image to be processed;

[0142] When the target matching tag is at the level of the parent tag, matching processing is performed in a preset image library based on the target matching tag to obtain a set of sub-tag images, and the target tag of the image to be processed is determined based on the set of sub-tag images.

[0143] Optionally, the preset image library includes at least one image, and each image includes an image tag. The processor 401 performs matching processing based on the target matching tag in the preset image library to obtain a sub-tag image set, including:

[0144] Calculate the correlation between the target matching tag and each of the image tags;

[0145] Based on the relevance, select the sub-tag image corresponding to the target matching tag from the preset image library;

[0146] Based on the sub-label images, the sub-label image set is constructed.

[0147] Optionally, the process of determining the target label of the image to be processed based on the sub-label image set, executed by the processor 401, includes:

[0148] Based on a preset image search engine, the target image is determined from the sub-tag image set;

[0149] The tags of the target image are used as the target tags of the image to be processed.

[0150] Optionally, the processor 401 executes a preset image search engine to determine the target image in the sub-tag image set, including:

[0151] The sub-tag image set and the image to be processed are provided to the preset image search engine so that the image search engine can calculate the image similarity between each sub-tag image in the sub-tag image set and the image to be processed.

[0152] Based on the image similarity, the target image is determined from the sub-label image set.

[0153] This invention also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the tag data processing method or the application-side tag data processing method provided in this invention, and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0154] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The computer-readable storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0155] The above description discloses only preferred embodiments of the present invention and should not be construed as limiting the scope of the present invention. Therefore, equivalent variations made in accordance with the claims of the present invention are still within the scope of the present invention.

Claims

1. A method for processing tag data, characterized in that, The method includes the following steps: Obtain the label data of the image to be processed from a multimodal large model; Based on the tag data, a matching process is performed in a preset tag library to obtain the matching result corresponding to the tag data. The preset tag library includes tags at multiple levels. Based on the hierarchy of the matching results, the target label of the image to be processed is determined.

2. The tag data processing method as described in claim 1, characterized in that, Before acquiring the label data of the multimodal large model for the image to be processed, the method further includes: Retrieve multiple preset tags; Among the multiple preset tags, multiple parent tags are determined; Each parent label is used as a classification target. Among the multiple preset labels, the child label corresponding to each parent label is determined, and the parent label has a higher level than the child label. Based on the parent tag and the child tags corresponding to the parent tag, the preset tag library is constructed.

3. The tag data processing method as described in claim 1, characterized in that, The matching result includes at least one matching tag, and each matching tag includes a matching confidence score with the tag data. Determining the target tag of the image to be processed based on the hierarchy of the matching result includes: Based on the matching confidence, the target matching label is selected from the matching results; Based on the hierarchy of the target matching tags, the target tags of the image to be processed are determined.

4. The tag data processing method as described in claim 3, characterized in that, The target matching tag is hierarchically defined as either a parent tag or a child tag. Determining the target tag of the image to be processed based on the hierarchy of the target matching tags includes: When the target matching tag is at the level of the sub-tag, the target matching tag is used as the target tag of the image to be processed; When the target matching tag is at the level of the parent tag, matching processing is performed in a preset image library based on the target matching tag to obtain a set of sub-tag images, and the target tag of the image to be processed is determined based on the set of sub-tag images.

5. The tag data processing method as described in claim 4, characterized in that, The preset image library includes at least one image, and each image includes an image tag. The matching process based on the target matching tag in the preset image library to obtain a sub-tag image set includes: Calculate the correlation between the target matching tag and each of the image tags; Based on the relevance, select the sub-tag image corresponding to the target matching tag from the preset image library; Based on the sub-label images, the sub-label image set is constructed.

6. The tag data processing method as described in claim 4, characterized in that, The step of determining the target tag of the image to be processed based on the sub-tag image set includes: Based on a preset image search engine, the target image is determined from the sub-tag image set; The tags of the target image are used as the target tags of the image to be processed.

7. The tag data processing method as described in claim 6, characterized in that, The image search engine, based on a preset formula, determines the target image from the sub-tag image set, including: The sub-tag image set and the image to be processed are provided to the preset image search engine so that the image search engine can calculate the image similarity between each sub-tag image in the sub-tag image set and the image to be processed. Based on the image similarity, the target image is determined from the sub-label image set.

8. A tag data processing device, characterized in that, The tag data processing device includes: The first acquisition module is used to acquire the label data of the multimodal large model for the image to be processed; The first matching module is used to perform matching processing in a preset tag library based on the tag data to obtain the matching result corresponding to the tag data. The preset tag library includes tags at multiple levels. The first determining module is used to determine the target label of the image to be processed based on the hierarchy of the matching results.

9. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the tag data processing method as claimed in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the tag data processing method as described in any one of claims 1 to 7.

Images