Information generation method and device, equipment, medium and program product

Abstract

The invention discloses an information generation method and device, equipment, a medium and a program product, and relates to the technical field of artificial intelligence, in particular to the technical fields of computer vision, deep learning, large models and the like. According to the specific implementation scheme, at least two types of label generation models are utilized, key information extraction is conducted on target objects, candidate label sets for the target objects are obtained, and the target objects comprise at least one of a target image, a target video and a target text; performing semantic consistency processing on the candidate label set according to an analysis result obtained by performing semantic consistency analysis on the candidate label set to obtain a target label set; and performing information generation processing on the target label set and the target object by using the multi-modal large model to obtain description information for the target object.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of artificial intelligence, in particular to the technical field of computer vision, deep learning, large model, etc., and more particularly to an information generation method, device, equipment, medium and program product. BACKGROUND

[0002] In image annotation, video content summary, text summary generation and other scenarios, generating accurate and comprehensive description information based on target objects is a core requirement. Traditional description information generation methods mostly rely on single model to extract feature labels, and have problems such as incomplete coverage of key information, resulting in deviation of the generated description content from the actual features of the target object. SUMMARY

[0003] The present disclosure provides an information generation method, device, equipment, medium and program product.

[0004] According to an aspect of the present disclosure, an information generation method is provided, comprising: using at least two types of label generation models to respectively extract key information of a target object to obtain a candidate label set for the target object, wherein the target object comprises at least one of the following: a target image, a target video, and a target text; performing semantic consistency processing on the candidate label set according to an analysis result obtained by performing semantic consistency analysis on the candidate label set to obtain a target label set; and using a multi-modal large model to perform information generation processing on the target label set and the target object to obtain description information for the target object.

[0005] According to another aspect of the present disclosure, an information generation device is provided, comprising: an information extraction module configured to use at least two types of label generation models to respectively extract key information of a target object to obtain a candidate label set for the target object, wherein the target object comprises at least one of the following: a target image, a target video, and a target text; a semantic consistency processing module configured to perform semantic consistency processing on the candidate label set according to an analysis result obtained by performing semantic consistency analysis on the candidate label set to obtain a target label set; and an information generation module configured to use a multi-modal large model to perform information generation processing on the target label set and the target object to obtain description information for the target object.

[0006] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method as described above.

[0007] According to another aspect of the present disclosure, there is provided a non-transitory computer readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to perform the method as described above.

[0008] According to another aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the method as described above.

[0009] It should be understood that the description of the contents of this section is not intended to identify key or essential features of embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become readily apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0010] The accompanying drawings are used to better understand the present scheme, and do not constitute a limitation on the present disclosure. Among them:

[0011] Figure 1 An exemplary system architecture to which the information generation and apparatus according to embodiments of the present disclosure can be applied is schematically shown;

[0012] Figure 2 A flowchart of an information generation method according to embodiments of the present disclosure is schematically shown;

[0013] Figure 3A A data flow diagram of generating a candidate label set according to embodiments of the present disclosure is schematically shown;

[0014] Figure 3B A flowchart of determining a candidate label set from an initial label set according to embodiments of the present disclosure is schematically shown;

[0015] Figure 4 A data flow diagram of determining a target label set according to embodiments of the present disclosure is schematically shown;

[0016] Figure 5 A data flow diagram of an information generation method according to embodiments of the present disclosure is schematically shown;

[0017] Figure 6 A block diagram of an information generation apparatus according to embodiments of the present disclosure is schematically shown; and

[0018] Figure 7 A block diagram of an electronic device suitable for implementing an information generation method according to embodiments of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0019] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0020] While there are ways for multiple models to collaboratively determine descriptive information in related technologies, there is still a lack of effective label filtering and semantic consistency processing mechanisms. The quality of label sets varies, making it difficult to support high-quality information generation.

[0021] Meanwhile, multimodal large models are highly sensitive to the semantic consistency and validity of input labels when generating descriptive information. Low-quality label input can easily lead to logical inconsistencies and missing details in the generated content. In addition, related technologies lack quality monitoring mechanisms for the generation process, making it impossible to identify problems such as excessive label selection and abnormal target object features in a timely manner, resulting in unstable quality of the generated descriptive information.

[0022] In view of this, embodiments of the present disclosure provide an information generation method, comprising: using at least two types of tag generation models to extract key information from a target object to obtain a candidate tag set for the target object, wherein the target object includes at least one of the following: target image, target video, and target text; performing semantic consistency processing on the candidate tag set based on the analysis results obtained from semantic consistency analysis of the candidate tag set to obtain a target tag set; and using a multimodal large model to perform information generation processing on the target tag set and the target object to obtain descriptive information for the target object.

[0023] Figure 1 An exemplary system architecture for information generation and apparatus that can be applied according to embodiments of this disclosure is illustrated.

[0024] It is important to note that Figure 1 The examples shown are merely examples of system architectures applicable to embodiments of this disclosure, intended to help those skilled in the art understand the technical content of this disclosure. They do not imply that embodiments of this disclosure cannot be used in other devices, systems, environments, or scenarios. For instance, in another embodiment, an exemplary system architecture to which the information generation method and apparatus can be applied may include a terminal device. However, the terminal device can implement the information generation method and apparatus provided in the embodiments of this disclosure without interacting with a server.

[0025] like Figure 1As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, and 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the terminal devices 101, 102, and 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0026] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as knowledge reading applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software, etc. (for example only).

[0027] Terminal devices 101, 102, and 103 can be various electronic devices with displays and web browsing capabilities, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0028] Server 105 can be a server that provides various services, such as a backend management server that supports the content browsed by users using terminal devices 101, 102, and 103 (for example only). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0029] It should be noted that the information generation method provided in this embodiment can generally be executed by terminal devices 101, 102, or 103. Correspondingly, the information generation device provided in this embodiment can also be disposed in terminal devices 101, 102, or 103.

[0030] Alternatively, the information generation method provided in this embodiment can generally be executed by server 105. Correspondingly, the information generation apparatus provided in this embodiment can generally be located in server 105. The information generation method provided in this embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with terminal devices 101, 102, 103 and / or server 105. Correspondingly, the information generation apparatus provided in this embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with terminal devices 101, 102, 103 and / or server 105.

[0031] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0032] In the technical solution disclosed herein, the collection, storage, use, processing, transmission, provision, disclosure, and application of user personal information comply with the provisions of relevant laws and regulations, necessary confidentiality measures have been taken, and there is no violation of public order and good morals.

[0033] In the technical solution disclosed herein, the user's authorization or consent is obtained before acquiring or collecting the user's personal information.

[0034] Figure 2 A flowchart illustrating an information generation method according to an embodiment of the present disclosure is shown schematically.

[0035] like Figure 2 As shown, the method includes operations S210~S230.

[0036] In operation S210, at least two types of label generation models are used to extract key information from the target object to obtain a set of candidate labels for the target object.

[0037] In operation S220, based on the analysis results obtained from the semantic consistency analysis of the candidate tag set, the candidate tag set is subjected to semantic consistency processing to obtain the target tag set.

[0038] In operation S230, using a multimodal large model, information generation processing is performed on the target label set and target object to obtain descriptive information for the target object.

[0039] The target object may include at least one of the following: target image, target video, or target text. The tag generation model can be used to extract key information from the target object to obtain a candidate tag set consisting of at least one tag for the target object, wherein the tags can be determined based on the core content and thematic features of the target object.

[0040] Specifically, when the target object is a target image, its label can refer to elements such as objects and scenes appearing in the target image.

[0041] It is understandable that a label generation model can at least handle data of the modality to which the target object belongs. For example, when the target object is a video, the label generation model can handle data of the video modality.

[0042] In particular, when the label generation model can only process data of a specific modality, the target object can be preprocessed to transform it into a modality that the label generation model can process, and then the label generation model can be used to extract key information.

[0043] Taking the example that the label generation model can only process image data, for target text, a text-to-image model can be used to process it to obtain the target image corresponding to the target text. For target video, video frame sampling can be used to obtain the target image corresponding to the target video, and the target image can be processed into a standardized format through operations such as image size normalization and format conversion. Thus, the consistency of the data input to the label generation model is ensured through strict preprocessing.

[0044] In this example, the target image can be scaled or cropped by normalizing the image size to obtain a standard image of 224×224 pixels. The format of the target image can also be unified by format conversion, such as converting images in BMP, JPG and other formats to PNG format.

[0045] Semantic consistency refers to the state in which, within the context of a given target object, there are no logical contradictions or conflicts in meaning among the tags in its candidate tag set, and they are compatible and supportive of each other in terms of topic and attribute description, forming a coherent whole.

[0046] Therefore, semantic consistency analysis can include semantic contradiction analysis and semantic overlap analysis to obtain analysis results. Semantic contradiction analysis means finding label pairs or groups that are semantically directly conflicting, while semantic overlap analysis means finding label pairs or groups that are semantically highly similar or have an inclusion relationship. The analysis results can include the aforementioned label pairs or groups that are semantically directly conflicting, as well as label pairs or groups that are semantically highly similar or have an inclusion relationship.

[0047] For example, semantic contradiction analysis can yield label groups that conflict semantically: "sunny" and "rainy". Semantic overlap analysis can yield label groups that are highly similar semantically or have an inclusion relationship: "person in yellow clothes" and "woman in yellow clothes".

[0048] Semantic consistency analysis can be performed using a large language model for processing text information. This can be done by inputting a set of candidate tags into the large language model and inputting the prompt "determine whether there are semantically contradictory or semantically repetitive tags in the tag set. If there are tags that meet the above conditions, filter out the tags and output them".

[0049] Based on the tag groups included in the analysis results, semantic consistency processing can be performed on the candidate tag set to obtain the target tag set. In the case of semantic conflicts between tag groups, multiple tags in the tag group can be removed from the candidate tag set to avoid contamination of the target tag set by uncertain tags. In the case of highly similar or inclusive tag groups, multiple tags in the tag group can be merged into a summary tag, and multiple tags in the tag group can be removed from the candidate tag set.

[0050] Multimodal large models can process data of multiple modalities. Target label sets and target objects can be input into the multimodal large model, which will then generate descriptive information for the target objects.

[0051] Specifically, prompt words can be generated based on the target tag set to inform the multimodal large model of the core content of the target object, and the multimodal large model can be required to generate descriptive information about the target object around the core content.

[0052] For example, for a target image, the resulting target label set includes three labels: "red car," "highway," and "sunny day." The generated prompt could be: "This image contains a red car and a highway. The environment shown in the image is a sunny day. Based on this, and combined with the image content, provide a more detailed description of the image."

[0053] Multimodal large models can rewrite sentences based on target labels in a target label set. During the rewriting process, the word order can be adjusted according to the confidence level of each target label, making the description information highlight high-confidence labels. Modifiers can also be added during the rewriting process to ensure the fluency and readability of the description information. Taking the target label set mentioned above as an example, after being rewritten by a multimodal large model, the output description information could be: A red car is driving on a wide highway under a clear sky.

[0054] According to embodiments of this disclosure, a multi-type label generation model is used to extract candidate labels, and semantic consistency processing and a multimodal large model are combined to generate descriptive information, achieving accurate and standardized generation of target object descriptive information. Compared with single-model label extraction schemes, multi-model collaboration improves the comprehensiveness of key information coverage. Through semantic consistency processing, contradictions and redundancies between labels can be eliminated, providing high-quality input for information generation. By using a multimodal large model to combine labels and the original target object, descriptive content that fits the target features is generated, solving the problems of traditional generation methods easily omitting key information and descriptions deviating from the topic.

[0055] According to embodiments of this disclosure, in Figure 2In the operation S210 shown, key information of the target object is extracted using at least two types of label generation models to obtain a candidate label set. This includes: extracting key information of the target object using at least two types of label generation models to obtain at least two initial label sets; and determining initial labels that meet the preset reliability conditions from the at least two initial label sets to obtain a candidate label set.

[0056] Figure 3A A data flow diagram illustrating the generation of a candidate tag set according to an embodiment of the present disclosure is shown schematically.

[0057] like Figure 3A As shown, three types of label generation models are selected: the first label generation model M310, the second label generation model M320, and the third label generation model M330. These label generation models can be any pre-trained large model, as long as they can process data corresponding to the modality of the target object.

[0058] Three label generation models are used to extract key information from the target object 310, resulting in a first initial label set 320, a second initial label set 330, and a third initial label set 340, respectively. Initial labels from these sets are filtered using pre-set reliability conditions, and those that meet the pre-set reliability conditions are added to the candidate label set 350.

[0059] For each tag generation model, a pre-set upper limit for the number of tags in the initial tag set can be configured. For example, if the upper limit for the number of tags in a tag generation model is set to 10, then after extracting key information from the target object using the tag generation model, the extraction results can be sorted by confidence level, and the 10 tags with the highest confidence level can be selected as the initial tag set. If the number of extracted results is less than the upper limit for the number of tags, all extracted results can be placed into the initial tag set.

[0060] During the key information extraction process, a model confidence level can be pre-set for each label generation model. The extracted results can then be filtered based on this model confidence level, with those meeting the required confidence level placed into the initial label set. It's important to note that the model confidence level here refers to the confidence level of the label generation model when generating labels, which differs from the pre-set confidence level of the candidates.

[0061] In one example, the pre-set confidence conditions can be quantified by a confidence threshold, which can be set to 0.8. The initial labels in the initial label set can be filtered directly by the confidence threshold, adding the initial labels with a confidence greater than 0.8 to the candidate label set and removing the initial labels with a confidence less than 0.8.

[0062] According to embodiments of this disclosure, initial tag sets are determined using at least two types of tag generation models, and further screening is performed to determine candidate tag sets. This makes the tags in the candidate tag sets more diverse and improves their usability. By selecting initial tags that meet confidence criteria to obtain the candidate tag set, pre-control of tag quality is achieved. This avoids errors in subsequent semantic analysis and information generation caused by low-confidence tags being mixed into the candidate set, further improving the reliability of candidate tags and laying a high-quality data foundation for semantic consistency processing.

[0063] In another example, a confidence threshold can be used to comprehensively evaluate the initial labels in at least two initial label sets. Specifically, determining the initial labels that meet the preset confidence conditions from at least two initial label sets to obtain a candidate label set may include: determining the average confidence of the initial labels based on their label confidence and the frequency of their occurrence in the at least two initial label sets; and determining the initial labels from the at least two initial label sets whose average confidence is greater than or equal to the preset confidence conditions to obtain a candidate label set.

[0064] The frequency of an initial label in at least two initial label sets can be determined by iterating through at least two initial label sets. Based on the frequency of an initial label in at least two initial label sets, and the same number of label confidences as the frequency of occurrence, the mean of the label confidences can be calculated as the average confidence.

[0065] For example, in the initial label set, the initial label "vehicle - red car" appears three times with confidence levels of 0.96, 0.9, and 0.9 respectively. Therefore, the average confidence level of this initial label is (0.96 + 0.9 + 0.9) / 3 = 0.92. Similarly, the initial label "road - highway" appears twice with confidence levels of 0.65 and 0.35 respectively. Therefore, the average confidence level of this initial label is (0.65 + 0.35) / 2 = 0.5.

[0066] The initial tag can include at least two sub-tags at different tag levels, where tag level indicates the level of detail in the tag description. For example, the lower the tag level of a sub-tag, the broader the content it describes; the higher the tag level, the more detailed the content it describes.

[0067] In one example, the initial label may include four sub-labels at different levels, ordered from lowest to highest: animal-canine-dog-Routi. This demonstrates that the higher the sub-label's level, the more detailed its description and the smaller its scope of application. The information generation method may further include: weighted fusion of the sub-label confidence scores based on their respective confidence scores and their weights, to obtain the initial label's label confidence score.

[0068] Because the higher the tag level of a sub-tag, the more detailed its description and the smaller its scope of application, and because the description of a sub-tag with a relatively low tag level can cover the description of a sub-tag with a relatively high tag level, generally speaking, the higher the tag level, the lower the corresponding confidence level.

[0069] You can set corresponding confidence weights for sub-tags at different tag levels, and then perform weighted fusion of the confidence scores of multiple sub-tags based on their confidence weights to obtain the tag confidence score of the initial tag.

[0070] Taking the four sub-tags at the above tag levels as an example, in order of tag level from low to high, the sub-tag confidence scores of the four sub-tags are 0.9, 0.85, 0.75, and 0.5 respectively, and the confidence weights are 0.6, 0.2, 0.15, and 0.05 respectively. Then, by weighting and fusing the sub-tag confidence scores of the multiple sub-tags, the tag confidence score is 0.8475.

[0071] According to embodiments of this disclosure, an initial label confidence score is obtained by fusing sub-label confidence scores with weighted averages, achieving refined calculation of multi-level label confidence scores. Differentiated weighting for sub-labels of varying granularity allows the label confidence scores to better reflect their actual representational capabilities, avoiding the bias inherent in coarse-grained label confidence score calculations and improving the comprehensiveness and reliability of label selection.

[0072] Figure 3B A flowchart illustrating the determination of a candidate tag set from an initial tag set according to an embodiment of the present disclosure is shown.

[0073] like Figure 3B As shown, the first initial label set 320, the second initial label set 330 and the third initial label set 340 can be summarized to obtain the summarized initial label set 360.

[0074] In operation S301, the initial label is retrieved from the aggregated initial label set, resulting in initial label 370.

[0075] In operation S302, based on the occurrence frequency and label confidence of the initial label 370 obtained from the aggregated initial label set 360, the average confidence of the initial label is calculated to obtain the average confidence 380.

[0076] For example, if there are three initial label sets, and a certain initial label appears twice in the three initial label sets, the label confidence scores for the two occurrences of the initial label can be determined as 0.85 and 0.95 respectively. The average label confidence score can be calculated based on this, and the average score of 0.9 can be taken as the average confidence score of the initial label.

[0077] In operation S303, determine whether the average confidence score 380 of the initial label 370 is greater than the confidence score threshold. If so, add the initial label 370 to the candidate label set 350; otherwise, execute operation S304.

[0078] In operation S304, the initial label 370 is removed from the aggregated initial label set 360.

[0079] According to embodiments of this disclosure, the average confidence level is calculated by combining label confidence and cross-model occurrence frequency, thereby achieving a quantitative determination of label validity. Compared to screening methods that solely rely on confidence, introducing occurrence frequency highlights the value of multi-model consensus labels, reduces interference from misjudged labels by a single model, and further improves the accuracy of candidate label screening.

[0080] According to embodiments of this disclosure, in Figure 2 In the operation S220 shown, semantic consistency processing of the candidate label set may include: in response to the analysis result including a semantically contradictory label group, removing semantically contradictory labels from the semantically contradictory label group from the candidate label set.

[0081] A semantically contradictory tag group can refer to a group of tags that are semantically directly conflicting. For example, when referring to the same subject in a target object, the tags "cat", "dog", and "rabbit" are semantically directly conflicting, so a semantically contradictory tag group includes the tags "cat", "dog", and "rabbit".

[0082] If the analysis results include semantically contradictory label groups, multiple semantically contradictory labels belonging to the semantically contradictory label groups can be identified from the candidate label set, and the semantically contradictory labels can be removed from the candidate label set to reduce the probability of errors in the labels in the candidate label set and ensure that the labels in the candidate label set are reliable and stable.

[0083] According to embodiments of this disclosure, similar to the initial label, the candidate label may also include at least two sub-labels at the label level, where the label level characterizes the level of detail in the label description; removing semantically contradictory labels from the semantically contradictory label group from the candidate label set may include: for any semantically contradictory label in the semantically contradictory label group, removing any semantically contradictory sub-labels from the candidate label set.

[0084] A semantically contradictory tag group can contain multiple semantically contradictory tags. In each semantically contradictory tag group, the lower-level sub-tags of multiple semantically contradictory tags can be consistent, while semantic contradictions exist in the higher-level sub-tags.

[0085] For example, a set of semantically contradictory tags are "feline-leopard" and "feline-leopard". These semantically contradictory tags are consistent in the first-level sub-tags, but there is a semantic contradiction in the second-level sub-tags.

[0086] In this situation, the confidence level of consistent sub-labels in semantically contradictory labels is relatively high. If the semantically contradictory labels are deleted directly as a whole, it will affect the comprehensiveness of the candidate label set, and thus affect the accuracy of subsequent information generation.

[0087] Therefore, for any semantically contradictory tag in a semantically contradictory tag group, we can first determine that the semantically contradictory tag has semantically contradictory sub-tags, and then remove the sub-tags from the candidate tag set.

[0088] According to embodiments of this disclosure, for semantic conflicts in multi-level tags, only the conflicting sub-tags are removed instead of the entire tag, achieving refined utilization of tag resources. This avoids the waste of discarding entire valid tags due to local sub-tag conflicts, and while eliminating semantic conflicts, it preserves the effective information in the tags to the greatest extent, improving the utilization rate of the initial tags.

[0089] Removing semantically contradictory tags after a semantic contradiction occurs can increase the probability that the remaining tags are all trustworthy tags. However, directly removing semantically contradictory tags or all sub-tags within semantically contradictory tags will result in the candidate tag set lacking key information, affecting the accuracy of the generated description information.

[0090] For example, after extracting key information from the same target object using different label generation models, the initial labels "sunny" and "rainy" are obtained respectively to describe the weather environment. The above initial labels have semantic contradictions and therefore belong to the semantically contradictory label group. According to the above operation, all semantically contradictory labels in the semantically contradictory label group will be removed, resulting in no labels for describing the weather environment of the target object in the candidate label set. The description information generated by the multimodal large model will also lack the description of the weather environment.

[0091] According to embodiments of this disclosure, in Figure 2 In the operation S220 shown, the semantic consistency processing of the candidate label set may further include: in response to the analysis result including a semantically contradictory label group, determining the label to be removed from the semantically contradictory label group based on the label confidence of each semantically contradictory label in the semantically contradictory label group, and removing the label to be removed from the candidate label set.

[0092] If the analysis results include semantically contradictory label groups, the semantically contradictory label with the highest label confidence can be selected and retained based on the label confidence of each semantically contradictory label in the semantically contradictory label group, while the other semantically contradictory labels are identified as labels to be removed.

[0093] For example, a semantically contradictory label group includes three contradictory labels: "cat," "dog," and "rabbit," with confidence levels of 0.7, 0.9, and 0.4, respectively. We can then retain the semantically contradictory label with the highest confidence level, "dog," and remove the other contradictory labels. This ensures that after removing the contradictory labels, the remaining semantically contradictory labels have the highest relative confidence levels, thus guaranteeing the reliability of the labels after semantic consistency processing.

[0094] In some embodiments, if the analysis results include a group of semantically contradictory tags, the semantically contradictory tag with the highest frequency of occurrence can be selected and retained, and the other semantically contradictory tags can be identified as tags to be removed, based on the frequency of occurrence of each of the multiple semantically contradictory tags in the semantically contradictory tag group.

[0095] For example, if a semantically contradictory tag group includes three semantically contradictory tags: "cat," "dog," and "rabbit," with frequencies of 10, 1, and 2 respectively, then the most frequent semantically contradictory tag, "cat," can be retained, while the other semantically contradictory tags are designated as tags to be removed.

[0096] According to embodiments of this disclosure, low-confidence labels in contradictory label groups are determined and removed based on their confidence levels, thus achieving the optimal retention of contradictory labels. Compared to the method of indiscriminately removing contradictory labels, prioritizing the retention of high-confidence labels ensures the information density and effectiveness of the candidate label set, further improving the quality of subsequent information generation.

[0097] According to embodiments of this disclosure, in Figure 2 In the operation S220 shown, the semantic consistency processing of the candidate label set may further include: in response to the analysis result including a label group to be fused, performing semantic fusion on multiple labels to be fused in the label group to be fused to obtain a fused label, wherein the label group to be fused contains multiple labels to be fused with overlapping semantics; and replacing the label group to be fused in the candidate label set with the fused label.

[0098] For example, semantic fusion is performed on multiple tags in a tag group to be fused, resulting in a fused tag that can cover the semantics of each of the multiple tags in the tag group. Then, the tag group to be fused is removed from the candidate tag set, and the fused tag is added to the candidate tag set.

[0099] For example, if the tag group to be merged includes three tags, "Maine Coon cat lying down" and "cat lying down", then semantic fusion can be performed on multiple tags to obtain a merged tag that covers the semantics of each of the multiple tags, namely "a cat lying down". The tag group to be merged in the candidate tag set is then replaced with the merged tag. In other words, the two tags "Maine Coon cat lying down" and "cat lying down" can be removed from the candidate tag set, and the merged tag "a cat lying down" can be added to the candidate tag set.

[0100] According to embodiments of this disclosure, semantically overlapping groups of tags to be fused are identified and semantically fused to generate a unified fused tag, thereby achieving redundancy removal of candidate tags. Merging duplicate or highly similar tags simplifies the size of the tag set, avoids the problem of repetitive and redundant descriptive information caused by redundant tags, and improves the simplicity and accuracy of information generation.

[0101] According to embodiments of this disclosure, semantic fusion of tags in a tag group to be fused to obtain a fused tag may specifically include: determining a target tag to be fused from multiple tags to be fused based on the frequency or confidence level of each tag to be fused in at least two initial tag sets, and then determining the target tag to be fused as the fused tag. In other words, the tag to be fused with the highest frequency in the initial tag set can be determined as the target tag to be fused, and further used as the fused tag.

[0102] For example, in another example, there are four initial label sets, and the label group to be merged includes two labels to be merged: "red car" and "red sedan". We can count the frequency of these labels in each of the four initial label sets. If "red car" appears three times and "red sedan" appears once, then "red car" can be identified as the merging label. In other words, in the target label set, "red car" is retained, and "red sedan" is removed.

[0103] According to embodiments of this disclosure, the core content of the fusion label is determined based on its frequency of occurrence across models, thereby enhancing the representativeness of the fusion label. High-frequency labels are preferentially selected as the fusion benchmark to ensure that the fusion label reflects the consensus characteristics of multiple models, improving the rationality and universality of the fusion label and providing more accurate guidance for information generation.

[0104] Figure 4 A data flow diagram illustrating the determination of a target tag set according to an embodiment of the present disclosure is shown schematically.

[0105] like Figure 4 As shown, by performing semantic consistency analysis on the candidate tag set 350, we can obtain the tag group 410 to be merged, the semantically contradictory tag group 420 and the ordinary tag 430. Among them, the ordinary tag 430 includes tags that do not have semantic contradictions or semantic overlaps.

[0106] By performing semantic fusion on multiple tags to be fused in the tag group 410, a fused tag 411 can be obtained.

[0107] By processing multiple semantically contradictory tags in the semantically contradictory tag group 420, the tag to be removed 421 can be determined. The tag to be removed 421 is removed from the semantically contradictory tag group 420, resulting in the remaining retained tag 422.

[0108] The target tag set 440 can be determined based on the fusion tag 411, the retained tag 422, and the ordinary tag 430. This allows the replacement of the tag group 410 to be fused using the fusion tag 411, and the removal of the tag 421 to be removed from the semantically contradictory tag group 420.

[0109] Because some information is discarded during the label removal and fusion process, or because the label generation model ignores some information in the initial generation stage, the descriptive information may not provide a complete and accurate description of the target object. A label generation model can be used to further generate labels for the target object based on the descriptive information, thus supplementing the missing information in the initially generated labels and descriptions.

[0110] Specifically, the information generation method may also include: using at least two types of label generation models to extract key information from the target object and descriptive information, and determining supplementary labels for the target information, wherein the supplementary labels indicate the key information missing in the descriptive information; using a multimodal large model to perform information generation processing on the supplementary labels, target object and descriptive information to obtain target descriptive information for the target object.

[0111] The prompts can be rewritten, for example: "Please check if there are any other details in the target object besides this description." The prompts, description, and target object are then input into the label generation model for further key information extraction. Because the rewritten prompts are used, supplementary labels will not be generated based on the key information already included in the description during key information extraction.

[0112] The supplementary labels, target objects, and descriptive information are input into the multimodal large model. The large model is controlled by prompt words to rewrite the descriptive information based on the supplementary labels and target objects. While retaining the original content of the descriptive information, the content of the supplementary labels is added to the descriptive information in combination with the target object to obtain the target descriptive information.

[0113] According to embodiments of this disclosure, iterative optimization of the description content is achieved by extracting supplementary tags from multiple models and generating description information a second time. Key tags are added to address missing points in the initially generated description information, guiding the multimodal large model to improve the description content. This solves the problem of easily missing details in a single generation and enhances the completeness and richness of the description information.

[0114] According to embodiments of this disclosure, the information generation method may further include: determining the number of target tags in the target tag set, and obtaining at least two intersection ratios relative to the number of initial tags in at least two initial tag sets, wherein the at least two initial tag sets are obtained by extracting key information from the target object using at least two types of tag generation models respectively; and marking the target object with a pending review identifier in response to the intersection ratio meeting a preset threshold condition, so as to conduct quality review on the target object.

[0115] For each initial tag set, the intersection ratio can be obtained by using the ratio of the number of target tags in the target tag set to the number of initial tags in the initial tag set. The intersection ratio can be used to represent the proportion of tags in the initial tag set that are retained in the target tag set, that is, the proportion of reliable high-quality tags in the initial tag set.

[0116] The preset threshold condition can be set to 20%. That is, for an initial tag set with an intersection ratio of less than 20%, it can be determined that there is a lot of noise or key information that is not easy to identify in the initial tag set. Therefore, the target object has quality problems and can be marked with a pending review label so that the target object can be reviewed based on the pending review label.

[0117] In one example, a target object has 20 initial tags in its initial tag set and 3 target tags in its target tag set. The intersection ratio is 3 / 20=15%, which is lower than the preset threshold condition. Therefore, the target object is marked as pending review.

[0118] In another example, if the initial tag set of another target object contains 30 initial tags and its target tag set includes 9 target tags, then the intersection ratio is 9 / 30=30%, which is higher than the preset threshold condition, so there is no need to mark the target object with the pending review label.

[0119] According to embodiments of this disclosure, the intersection ratio between target tags and initial tags is calculated and a quality review is triggered, thereby enabling monitoring of the target object and tag generation quality. By quantifying the information retention rate during the tag screening process using the ratio indicator, target objects that require discarding too many initial tags due to low quality can be identified, marked, and reviewed, ensuring the overall quality of the target objects.

[0120] According to embodiments of this disclosure, the preset threshold condition includes a first threshold and a second threshold; in response to the intersection ratio meeting the preset threshold condition, marking the target object with a pending review identifier includes: in response to at least one intersection ratio being less than the first threshold, or the average of at least two intersection ratios being less than the second threshold, marking the target object with a pending review identifier.

[0121] In one example, if the proportion of the intersection of an initial tag set is less than a first threshold, it is determined that the proportion of high-quality tags in the initial tag set is low, and therefore the proportion of the intersection meets the preset threshold condition.

[0122] In another example, if the percentage of intersection in each initial tag set is less than a first threshold, it is determined that the percentage of high-quality tags in all initial tag sets is low, and therefore the percentage of intersection meets the preset threshold condition.

[0123] In another example, if the average of the proportions of at least two intersections is less than a second threshold, it is determined that the overall proportion of high-quality tags in multiple initial tag sets is low, and therefore the proportion of intersections can be determined to meet the preset threshold condition.

[0124] The first threshold and the second threshold can be the same or different. Preferably, the first threshold is less than the second threshold. For example, the first threshold can be 10% and the second threshold can be 20%. That is, if there is at least one initial tag set with an intersection ratio of less than 10%, it can be determined that the target object has an initial tag set with an excessively low intersection ratio, and therefore it is marked as pending review. If the average of multiple intersection ratios is less than 20%, it can be determined that the overall intersection ratio of the target object is low, and therefore it is marked as pending review.

[0125] According to embodiments of this disclosure, a dual-threshold condition is set to trigger the pending review flag, thereby achieving greater precision in quality review triggering. By using dual judgments—one where the single percentage is below the first threshold and the average percentage below the second threshold—both individual model label anomalies and overall label screening anomalies are taken into account, avoiding false triggers or missed triggers due to a single threshold judgment, and improving the comprehensiveness and accuracy of quality monitoring.

[0126] Figure 5 A data flow diagram of an information generation method according to an embodiment of the present disclosure is illustrated schematically.

[0127] like Figure 5 As shown, this embodiment uses three tag generation models as examples to illustrate the information generation method. The information generation method of this embodiment includes: inputting the target object 310 into the first tag generation model M310, the second tag generation model M320, and the third tag generation model M330 respectively, generating a first initial tag set 320, a second initial tag combination 330, and a third initial tag set 340 respectively; then, performing confidence screening on the first initial tag set 320, the second initial tag combination 330, and the third initial tag set 340 to obtain a candidate tag set 350; and performing semantic consistency processing on the candidate tag set 350 to obtain a target tag set 440. After determining the target tag set 440, for each tag generation model, the intersection ratio can be calculated based on the initial tag set and the target tag set respectively.

[0128] Figure 5 The diagram shows the third initial label set 340 generated by the third label generation model M330, and explains the intersection ratio.

[0129] Specifically, the intersection ratio 520 can be determined based on the ratio of the number of initial tags in the third initial tag set 340 to the number of target tags in the target tag set 440. Then, operation S501 is executed.

[0130] In operation S501, determine whether the intersection ratio meets the preset threshold condition. If yes, execute operation S502; otherwise, execute operation S503.

[0131] In operation S502, the target object 310 is marked with a pending audit mark.

[0132] In operation S503, description information for target object 310 is generated based on target tag set 440.

[0133] Figure 6 A block diagram of an information generation apparatus according to an embodiment of the present disclosure is shown schematically.

[0134] like Figure 6 As shown, the information generation device 600 of this embodiment includes an information extraction module 610, a semantic consistency processing module 620, and an information generation module 630.

[0135] The information extraction module 610 is used to extract key information from the target object using at least two types of label generation models, and obtain a set of candidate labels for the target object, wherein the target object includes at least one of the following: target image, target video, and target text.

[0136] The semantic consistency processing module 620 is used to perform semantic consistency processing on the candidate tag set based on the analysis results obtained from the semantic consistency analysis of the candidate tag set, so as to obtain the target tag set.

[0137] The information generation module 630 is used to generate information from the target label set and the target object using a multimodal large model, so as to obtain descriptive information for the target object.

[0138] According to embodiments of this disclosure, the information extraction module 610 includes an information extraction submodule and a first tag filtering submodule.

[0139] The information extraction submodule is used to extract key information from the target object using at least two types of label generation models, and obtain at least two initial label sets.

[0140] The first label filtering submodule is used to determine the initial labels that meet the preset reliability conditions from at least two initial label sets, and obtain the candidate label set.

[0141] According to embodiments of this disclosure, the first tag filtering submodule includes a confidence calculation unit and a tag filtering unit.

[0142] The confidence calculation unit is used to determine the average confidence of the initial label based on the label confidence of the initial label and the frequency of occurrence of the initial label in at least two initial label sets.

[0143] The label filtering unit is used to determine the initial labels with an average confidence level greater than or equal to a preset confidence level from at least two initial label sets, thereby obtaining a candidate label set.

[0144] According to embodiments of this disclosure, the information generation apparatus 600 further includes a confidence level determination module.

[0145] The confidence level determination module is used to perform weighted fusion of the sub-label confidence levels based on the sub-label confidence levels of each sub-label in the initial label and the confidence level weight of each sub-label to obtain the label confidence level of the initial label.

[0146] According to embodiments of this disclosure, the semantic consistency processing module 620 includes a tag removal submodule.

[0147] The tag removal submodule is used to remove semantically contradictory tags from the candidate tag set in response to analysis results including semantically contradictory tag groups.

[0148] According to embodiments of this disclosure, the label removal submodule includes a sublabel removal submodule.

[0149] The sub-tag removal submodule is used to remove sub-tags that contain semantic contradictions from the candidate tag set for any semantically contradictory tag in a semantically contradictory tag group.

[0150] According to embodiments of this disclosure, the semantic consistency processing module 620 includes a second tag filtering submodule.

[0151] The second label filtering submodule is used to respond to the analysis results including semantically contradictory label groups, determine the labels to be removed from the semantically contradictory label groups based on the label confidence of each semantically contradictory label in the semantically contradictory label group, and remove the labels to be removed from the candidate label set.

[0152] According to embodiments of this disclosure, the semantic consistency processing module 620 includes a tag fusion submodule and a tag replacement submodule.

[0153] The tag fusion submodule is used to perform semantic fusion on multiple tags in the tag group to be fused in response to the analysis results including the tag group to be fused, so as to obtain fused tags. The tag group to be fused contains multiple tags with semantic overlap.

[0154] The tag replacement submodule is used to replace the group of tags to be merged in the candidate tag set with the merged tag.

[0155] According to embodiments of this disclosure, the tag fusion submodule includes a tag fusion unit.

[0156] The tag fusion unit is used to determine the target tag to be fused from multiple tags to be fused based on the frequency of occurrence of each tag to be fused in at least two initial tag sets, and to determine the target tag to be fused as the fusion tag.

[0157] According to embodiments of this disclosure, the information generation apparatus 600 further includes a tag supplementation module and an information supplementation module.

[0158] The tag supplementation module is used to extract key information from the target object and descriptive information using at least two types of tag generation models, and to determine supplementary tags for the target information. The supplementary tags indicate the key information missing in the descriptive information.

[0159] The information supplementation module is used to generate information from supplementary labels, target objects, and descriptive information using a multimodal large model, thereby obtaining target descriptive information for the target object.

[0160] According to embodiments of this disclosure, the information generation apparatus 600 further includes a percentage calculation module and an object marking module.

[0161] The proportion calculation module is used to determine the ratio of the number of target tags in the target tag set to the number of initial tags in at least two initial tag sets, and to obtain the proportion of at least two intersections. The at least two initial tag sets are obtained by extracting key information from the target object using at least two types of tag generation models.

[0162] The object tagging module is used to tag target objects with a pending review identifier in response to the intersection ratio meeting a preset threshold condition, so as to conduct quality review of the target objects.

[0163] According to embodiments of this disclosure, the object tagging module includes an object tagging submodule.

[0164] The object tagging submodule is used to tag the target object with a pending review identifier in response to at least one intersection percentage being less than a first threshold, or the average of at least two intersection percentages being less than a second threshold.

[0165] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0166] According to an embodiment of the present disclosure, an electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method described above.

[0167] According to embodiments of the present disclosure, a non-transitory computer-readable storage medium stores computer instructions, wherein the computer instructions are used to cause a computer to perform the method described above.

[0168] According to an embodiment of this disclosure, a computer program product includes a computer program that, when executed by a processor, implements the method described above.

[0169] Figure 7 A block diagram schematically illustrates an electronic device 700 suitable for implementing an information generation method according to embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0170] like Figure 7 As shown, device 700 includes a computing unit 701, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 702 or a computer program loaded into random access memory (RAM) 703 from storage unit 708. The RAM 703 may also store various programs and data required for the operation of device 700. The computing unit 701, ROM 702, and RAM 703 are interconnected via bus 704. Input / output (I / O) interface 705 is also connected to bus 704.

[0171] Multiple components in device 700 are connected to input / output (I / O) interface 705, including: input unit 706, such as keyboard, mouse, etc.; output unit 707, such as various types of monitors, speakers, etc.; storage unit 708, such as disk, optical disk, etc.; and communication unit 709, such as network card, modem, wireless transceiver, etc. Communication unit 709 allows device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0172] The computing unit 701 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as information generation methods. For example, in some embodiments, the information generation method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the information generation method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform information generation methods by any other suitable means (e.g., by means of firmware).

[0173] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0174] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0175] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0176] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0177] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0178] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, distributed system servers, or servers incorporating blockchain technology.

[0179] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0180] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. An information generation method, comprising: Using at least two types of tag generation models, key information is extracted from the target object to obtain a set of candidate tags for the target object, wherein the target object includes at least one of the following: target image, target video, and target text; Based on the analysis results obtained from the semantic consistency analysis of the candidate tag set, semantic consistency processing is performed on the candidate tag set to obtain the target tag set; and Using a multimodal large model, information generation processing is performed on the target label set and the target object to obtain descriptive information for the target object.

2. The method according to claim 1, wherein, The method utilizes at least two types of label generation models to extract key information from the target object, thereby obtaining a candidate label set, including: Using the at least two types of tag generation models, key information is extracted from the target object to obtain at least two initial tag sets; From the at least two initial label sets, initial labels that satisfy the preset confidence conditions are determined to obtain the candidate label set.

3. The method according to claim 2, wherein, The step of determining initial labels that satisfy preset reliability conditions from the at least two initial label sets to obtain the candidate label set includes: The average confidence level of the initial label is determined based on the label confidence level of the initial label and the frequency of occurrence of the initial label in the at least two initial label sets; From the at least two initial label sets, initial labels with an average confidence level greater than or equal to the preset confidence condition are determined to obtain the candidate label set.

4. The method according to claim 3, wherein, The initial tag includes at least two tag levels of sub-tags, where the tag level characterizes the level of detail in the tag description; The method further includes: Based on the sub-label confidence scores of each sub-label in the initial label and the confidence scores of each sub-label, the sub-label confidence scores are weighted and fused to obtain the label confidence score of the initial label.

5. The method according to any one of claims 1 to 4, wherein, The semantic consistency processing of the candidate tag set includes: In response to the analysis results including a semantically contradictory tag group, semantically contradictory tags in the semantically contradictory tag group are removed from the candidate tag set.

6. The method according to claim 5, wherein, The candidate labels include at least two sub-labels at the label level, where the label level characterizes the level of detail in the label description. The step of removing semantically contradictory tags from the semantically contradictory tag group from the candidate tag set includes: For any semantically contradictory tag in the semantically contradictory tag group, remove the semantically contradictory sub-tags from the candidate tag set.

7. The method according to any one of claims 1 to 4, wherein, The semantic consistency processing of the candidate tag set includes: In response to the analysis results including a semantically contradictory label group, based on the label confidence of each semantically contradictory label in the semantically contradictory label group, a label to be removed is determined from the semantically contradictory label group, and the label to be removed is removed from the candidate label set.

8. The method according to any one of claims 1 to 7, wherein, The semantic consistency processing of the candidate tag set includes: In response to the analysis result including a group of tags to be merged, semantic fusion is performed on multiple tags in the group of tags to be merged to obtain merged tags, wherein the group of tags to be merged contains multiple tags with overlapping semantics; Replace the group of tags to be merged in the candidate tag set with the merging tag.

9. The method according to claim 8, wherein, The step of semantically fusing the tags in the tag group to be fused to obtain fused tags includes: The target tag to be merged is determined from the plurality of tags to be merged based on the frequency of occurrence of each of the tags to be merged in at least two initial tag sets, and the target tag to be merged is determined as the fusion tag.

10. The method according to any one of claims 1 to 9, further comprising: Using the at least two types of tag generation models, key information is extracted from the target object and the description information to determine supplementary tags for the target information, wherein the supplementary tags indicate the key information missing in the description information; Using the multimodal large model, information generation processing is performed on the supplementary label, the target object, and the descriptive information to obtain target descriptive information for the target object.

11. The method according to any one of claims 1 to 10, further comprising: Determine the ratio of the number of target tags in the target tag set to the number of initial tags in at least two initial tag sets, and obtain at least two intersection ratios, wherein the at least two initial tag sets are obtained by extracting key information from the target object using the at least two types of tag generation models respectively; In response to the intersection ratio meeting a preset threshold condition, the target object is marked with a pending review identifier so that the target object can be subject to quality review.

12. The method according to claim 11, wherein, The preset threshold conditions include a first threshold and a second threshold; The step of marking the target object with a pending review identifier in response to the intersection ratio meeting a preset threshold condition includes: In response to at least one intersection percentage being less than the first threshold, or the average of the percentages of the at least two intersections being less than the second threshold, the target object is marked with a pending review identifier.

13. An information generation device, comprising: The information extraction module is used to extract key information from the target object using at least two types of tag generation models, and obtain a set of candidate tags for the target object, wherein the target object includes at least one of the following: target image, target video, and target text; The semantic consistency processing module is used to perform semantic consistency processing on the candidate tag set based on the analysis results obtained from the semantic consistency analysis of the candidate tag set, to obtain the target tag set; and The information generation module is used to perform information generation processing on the target label set and the target object using a multimodal large model to obtain descriptive information for the target object.

14. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-12.

15. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-12.

16. A computer program product comprising a computer program stored on at least one of a readable storage medium and an electronic device, the computer program implementing the method according to any one of claims 1-12 when executed by a processor.

Images