Image-based cue word generation method and device, equipment and storage medium

Abstract

The invention relates to the technical field of computer vision and natural language processing, and discloses an image-based cue word generation method, which comprises the following steps of: obtaining non-text vision elements, text elements and position information thereof from a cue image, screening and processing the elements, removing text contents in the image and generating a cue image without characters. And based on the target non-text visual elements, cutting out corresponding image areas from the prompt image without characters, inputting the image areas into the multi-modal model, and generating prompt words related to the visual elements. According to the method, the non-text visual elements and the text elements in the image are automatically recognized, the image without characters is generated, and character interference is avoided. The cue word highly related to the image content is generated, the cue word generation accuracy and automation are achieved, and the processing efficiency and the cue word quality are improved. The complexity of writing the cue word by the user is effectively reduced, so that the user can generate the cue word with high quality and accurate semantics through simple image interaction.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and natural language processing, and in particular to an image-based prompt word generation method, device, equipment and storage medium. Background Art

[0002] In the field of computer vision, image generation technology has made significant progress in recent years, especially image generation methods based on diffusion models, which have surpassed traditional generative adversarial networks (GANs) in terms of diversity and image quality. Diffusion models generate images through a gradual denoising process, and the training process is more stable, so they show higher reliability and consistency in the actual application of image generation. This progress has led to the widespread application of image generation in fields such as promotion and marketing, creative design, etc.

[0003] In the financial services and insurance sectors, companies often need to design and produce a variety of images and texts to promote their insurance, secured loans and other products based on different business scenarios, time nodes, and regional characteristics. This type of customized image demand not only needs to meet the diversity of business, but also needs to respond quickly to market changes. However, traditional image generation methods require users to have a high level of image design and business demand understanding ability, especially in the writing of prompts, which greatly limits the scope of use for non-professional users.

[0004] Image generation technology based on diffusion model relies on the fusion of multimodal models, in which the text model provides semantic control capabilities for the generated results. Prompt, as the input text, determines the quality of the generated image and its fit with business needs. Despite this, writing prompts is still a complex task, requiring users to understand business needs and have knowledge of image design, such as composition, light and shadow, style, etc. In order to solve this problem, a prompt generation method based on interactive images is proposed. Through the multimodal interaction of images and texts, the user's usage threshold is lowered, and the effect of what you see is what you get is achieved, thereby improving the efficiency and quality of image generation. Summary of the invention

[0005] The main purpose of the present invention is to provide a method, device, equipment and storage medium for generating prompt words based on images, aiming to solve the technical problem of how to efficiently generate high-quality prompt words through simple image interaction.

[0006] To achieve the above object, the present invention provides a method for generating prompt words based on an image, comprising:

[0007] Acquire a prompt image, and identify non-text visual elements, text elements, and corresponding position information of the prompt image;

[0008] Filtering out target non-text visual elements from the non-text visual elements;

[0009] Based on the text element and the corresponding position information, remove the text in the prompt image to generate a prompt image without text;

[0010] A target image area is cropped from the image without text prompt based on the target non-text visual element and the corresponding position information, and the target image area is input into a multimodal model to obtain a prompt word based on the non-text visual element.

[0011] In one embodiment, after inputting the target image area into the multimodal model to obtain the prompt words based on the non-text visual elements, the method further includes:

[0012] Inputting the image without text prompt into a multimodal embedding model to obtain an image embedding representation;

[0013] Inputting the prompt words based on the non-text visual elements into a multimodal embedding model to obtain an embedding representation of each target non-text visual element;

[0014] A cosine distance between an embedding representation of each target non-text visual element and an embedding representation of the image is calculated, and a target prompt word is selected from the prompt words based on the non-text visual element based on the calculation result of the cosine distance.

[0015] In one embodiment, the step of selecting a target prompt word from the prompt words based on the non-text visual element based on the calculation result of the cosine distance includes:

[0016] Selecting a first preset number of preferred prompt words with the smallest calculation results of cosine distance from the prompt words based on the non-text visual element;

[0017] Randomly select a second preset number of secondary selected prompt words from the remaining prompt words based on non-textual visual elements except the first prompt word;

[0018] The target prompt word is obtained by combining the first prompt word and the second prompt word.

[0019] In one embodiment, after combining the first prompt word with the second prompt word to obtain the target prompt word, the method further includes:

[0020] Use the pre-trained semantic analysis model to semantically match and check the content of each target prompt word, and adjust the content of each target prompt word according to the semantic analysis results;

[0021] Obtain the user's historical selection data, identify the user's prompt word selection preference, and assign corresponding weights to each adjusted target prompt word;

[0022] Each target prompt word is scored according to the preset quality standard and the weight of each target prompt word, and the target prompt word with the highest score is recommended to the user.

[0023] In one embodiment, removing the text in the prompt image based on the text element and the corresponding position information to generate a prompt image without text includes:

[0024] Based on the text element and the corresponding position information, determining a text area in the prompt image that needs to be removed;

[0025] Performing transparent processing on the text area to be removed, removing the text element, and generating a blank area;

[0026] The pixel average value within a preset adjacent range of the removed text element is calculated, and the blank area generated after the text element is removed is filled with the pixel average value to generate the image without text prompt.

[0027] In one embodiment, after inputting the target image area into the multimodal model to obtain a prompt word based on a non-text visual element, the method further includes:

[0028] Performing a grammatical and logical consistency check on the prompt words based on the non-textual visual elements based on preset semantic rules, and if grammatical and / or logical errors are found, correcting the prompt words based on the non-textual visual elements;

[0029] A plurality of versions of candidate prompt words are generated according to the revised prompt words based on the non-textual visual element, and a final prompt word based on the non-textual visual element is selected from the plurality of candidate prompt words according to a preset selection criterion.

[0030] In one embodiment, the step of selecting a target non-text visual element from the non-text visual elements includes:

[0031] Determine whether an element retention instruction is detected;

[0032] If so, filtering out a target non-text visual element from the non-text visual elements according to the element retention instruction;

[0033] If not, a third preset number of target non-text visual elements are selected from the non-text visual elements.

[0034] Furthermore, to achieve the above-mentioned purpose, the present invention also provides an image-based prompt word generation device, comprising:

[0035] An image recognition module, used to obtain a prompt image, and recognize non-text visual elements, text elements, and corresponding position information of the prompt image;

[0036] A target screening module, used for screening out target non-text visual elements from the non-text visual elements;

[0037] A text removal module, used to remove the text in the prompt image based on the text element and the corresponding position information, and generate a text-free prompt image;

[0038] The prompt word generation module is used to crop a target image area from the text-free prompt image based on the target non-text visual element and the corresponding position information, input the target image area into the multimodal model, and obtain a prompt word based on the non-text visual element.

[0039] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a computer device, which includes a memory, a processor, and an image-based prompt word generation program stored in the memory and executable on the processor, wherein the image-based prompt word generation program implements the steps of the image-based prompt word generation method as described above when executed by the processor.

[0040] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a computer storage medium, on which is stored an image-based prompt word generation program, and when the image-based prompt word generation program is executed by a processor, the steps of the image-based prompt word generation method as described above are implemented.

[0041] Beneficial effects: The present invention relates to the technical field of computer vision and natural language processing, and discloses a method for generating prompt words based on an image. By obtaining non-text visual elements and text elements and their position information from a prompt image, these elements are screened and processed to remove text content in the image and generate a prompt image without text. Based on the target non-text visual elements, the corresponding image area is cropped from the prompt image without text, and it is input into a multimodal model to generate prompt words related to these visual elements. The present invention generates a text-free image by automatically identifying non-text visual elements and text elements in the image, thereby avoiding text interference. Prompt words that are highly relevant to the image content are generated, the accuracy and automation of prompt word generation are achieved, and the processing efficiency and prompt word quality are improved. The complexity of user prompt word writing is effectively reduced, so that users can generate high-quality and semantically accurate prompt words through simple image interaction. . BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0043] Figure 1 It is a flow chart of an embodiment of a method for generating prompt words based on an image according to the present invention;

[0044] Figure 2 It is a functional module diagram of a preferred embodiment of the image-based prompt word generation device of the present invention;

[0045] Figure 3 It is a structural diagram of the device hardware operating environment involved in the computer device embodiment of the present invention. DETAILED DESCRIPTION

[0046] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0047] It should be noted that in the field of computer vision, image generation technology has made significant progress in recent years, especially image generation methods based on diffusion models, which have surpassed traditional generative adversarial networks (GANs) in terms of diversity and image quality. Diffusion models generate images through a gradual denoising process, and the training process is more stable, so they show higher reliability and consistency in the actual application of image generation. This progress has led to the widespread application of image generation in fields such as promotion and marketing, creative design, etc.

[0048] In the financial services and insurance sectors, companies often need to design and produce a variety of images and texts to promote their insurance, secured loans and other products based on different business scenarios, time nodes, and regional characteristics. This type of customized image demand not only needs to meet the diversity of business, but also needs to respond quickly to market changes. However, traditional image generation methods require users to have a high level of image design and business demand understanding ability, especially in the writing of prompts, which greatly limits the scope of use for non-professional users.

[0049] Image generation technology based on diffusion model relies on the fusion of multimodal models, in which the text model provides semantic control capabilities for the generated results. Prompt, as the input text, determines the quality of the generated image and its fit with business needs. Despite this, writing prompts is still a complex task, requiring users to understand business needs and have knowledge of image design, such as composition, light and shadow, style, etc. In order to solve this problem, a prompt generation method based on interactive images is proposed. Through the multimodal interaction of images and texts, the user's usage threshold is lowered, and the effect of what you see is what you get is achieved, thereby improving the efficiency and quality of image generation.

[0050] See also Figure 1 , Figure 1 This is a flow chart of an embodiment of the image-based prompt word generation method provided by the present invention. It should be noted that although the logical order is shown in the flow chart, in some cases, the steps shown or described may be performed in a different order than that here.

[0051] like Figure 1 As shown, the image-based prompt word generation method proposed by the present invention includes the following steps:

[0052] S10, acquiring a prompt image, and identifying non-text visual elements, text elements, and corresponding position information of the prompt image;

[0053] In this embodiment, the prompt image refers to an image used to generate a prompt word (Prompt) or as a model input. The prompt image contains content that needs to be recognized and analyzed, including non-text visual elements (such as objects, scenes, etc.) and text elements (such as text information). These images are usually used in multimodal models to generate prompt words related to visual elements by recognizing and processing their content, and then used for specific image generation or analysis tasks.

[0054] Extract important information from the input prompt image, including objects in the image (non-text visual elements such as buildings and vehicles) and text content (text elements). By identifying these elements and their locations in the image, basic data support can be provided for subsequent image processing and analysis. This process usually involves semantic segmentation models (such as SAM Vit-L) and text detection and recognition models (such as PP-OCR), which can process non-text visual elements and text elements respectively to ensure accurate extraction of required information in complex images.

[0055] In one specific implementation, the system first extracts non-text visual elements and text elements from the prompt image. An advanced semantic segmentation model is used to identify and separate various objects in the image, while a text detection model is used to identify the text content in the image. The system records the location information of these elements to provide accurate data support for subsequent steps. In this implementation, the system is able to process multiple types of image inputs simultaneously, and can effectively identify and extract useful visual and text information, whether it is a natural scene or a complex urban street scene.

[0056] In another specific embodiment, the system can selectively identify specific types of non-text visual elements based on user needs and specific characteristics of the image. For example, the system can prioritize the identification of certain important objects, such as billboards, vehicles, etc., based on preset business scenarios, while ignoring irrelevant objects in the background. The identification of text elements can also be adjusted according to user needs, such as only identifying titles or key content. This implementation increases the flexibility of the system, enabling it to adapt to different application scenarios and meet diverse business needs.

[0057] In other specific embodiments, the system can further optimize the recognition process of visual elements and text elements by combining different models and algorithms. For example, the system can combine deep learning models and traditional image processing techniques to ensure that target elements can be accurately recognized and extracted under various lighting conditions and complex backgrounds. At the same time, the system can adopt a multi-stage recognition strategy, first roughly identifying elements and then performing fine processing in subsequent stages to improve the accuracy and robustness of recognition. This implementation is particularly suitable for image processing tasks with high precision requirements, such as medical image analysis or visual element recognition in autonomous driving scenarios.

[0058] Through the above steps, key visual and text information can be automatically and efficiently extracted from images, which not only improves the automation of image processing, but also adapts to a variety of application scenarios through flexible implementation methods. In addition, the system can maintain high-precision recognition capabilities in complex image environments, and both the main objects and text content in the image can be accurately extracted and located, significantly reducing the need for manual intervention.

[0059] S20, screening out target non-text visual elements from the non-text visual elements;

[0060] In this embodiment, the most relevant or pre-conditional elements are selected from all the identified non-text visual elements (e.g., objects, buildings, vehicles, etc.), which are referred to as "target non-text visual elements". The screening process can be based on multiple criteria, such as the importance score of the element, the relevance to the context, the user's specific needs, or pre-set business rules.

[0061] In one specific implementation, the system scores all identified non-text visual elements and automatically selects several target non-text visual elements with the highest scores according to their importance in the image or their matching degree with user needs. Such scoring criteria may include the area, position, color or shape of the element, and the semantic relevance to the overall image.

[0062] In another specific implementation, the user can manually select the desired non-text visual elements, and the system will mark these user-specified elements as target non-text visual elements. This method allows the user to perform a secondary screening of the identified visual elements to ensure that the final selected elements fully meet the user's needs. The system can also provide a user suggestion function to recommend possible target elements for the user to choose based on historical selection data.

[0063] In other specific implementations, the system can automatically filter target non-text visual elements based on preset rules or scenarios. For example, in a task of generating an advertising image, the system can prioritize billboards, trademarks, or other elements with brand logos as target non-text visual elements, while ignoring irrelevant objects in the background. This implementation is suitable for specific industries or application scenarios, ensuring that the generated content is highly relevant and meets business needs.

[0064] Through the above steps, the most relevant non-text visual elements can be effectively screened out in a complex image environment, ensuring that the subsequent processing and generation steps are based on the most valuable information. This not only improves the intelligence level of the system, but also adapts to the needs of different application scenarios through a flexible screening mechanism. In addition, users can participate in the screening process, which enhances the operability and practicality of the system and ultimately improves the relevance and accuracy of the generated content.

[0065] S30, removing the text in the prompt image based on the text element and the corresponding position information to generate a prompt image without text;

[0066] In this embodiment, the text in the image is identified and removed based on its position information, thereby generating an image without text interference. This process usually includes text detection, positioning, removal and filling of blank areas, so that the image looks complete and unaffected visually.

[0067] In one specific implementation, the system first uses a text recognition model (such as PP-OCR) to detect all text elements in the prompt image and accurately locate the positions of these text elements in the image. Then, the system removes these texts from the image based on the position information. The removal process can be performed by an overlay method, that is, directly filling the area of ​​the removed text with the pixel mean of the adjacent area to generate a prompt image without text. This method can ensure that the filled area blends naturally with the surrounding background.

[0068] In another specific implementation, in addition to using pixel mean filling, the system can also combine image restoration technology, such as a deep learning-based image restoration model, to intelligently restore the area where the text is removed. This method can further optimize the filling effect, making the removed image more visually harmonious and consistent, and less likely to detect traces of modification.

[0069] In other specific embodiments, the system can select an appropriate removal method according to different scenes of the image and the complexity of the text. For example, for a prompt image with a complex background, the system can use an advanced texture synthesis method to fill the removed text area more finely based on the texture information of the surrounding area. This method can generate a prompt image without text while maintaining the overall visual quality of the image, and is suitable for scenes with high-precision image processing requirements.

[0070] Through the above steps, the text content in the image can be effectively removed, and a text-free prompt image can be generated, providing a cleaner input for subsequent image processing and analysis. It not only improves the accuracy and automation of image processing, but also enables the flexible application of various technical means according to different scenarios to ensure image quality and consistency.

[0071] S40, cropping a target image area from the image without text prompt based on the target non-text visual element and the corresponding position information, inputting the target image area into a multimodal model, and obtaining a prompt word based on the non-text visual element.

[0072] In this embodiment, in the image with text removed, the corresponding image area is accurately cropped based on the previously screened target non-text visual elements and their position information. By inputting these image areas into a multimodal model (such as CLIP, BLIP), the system can generate prompt words that are closely related to these visual elements. This process not only involves image processing, but also combines advanced multimodal learning techniques to make the generated prompt words more semantically relevant and accurate.

[0073] In one specific implementation, the system crops the corresponding image area from the text-free prompt image based on the position information of the screened target non-text visual elements. This process first uses the position information to determine the bounding box of the target area, and then accurately crops the area to ensure that the cropped part contains all relevant visual elements. The cropped image area is directly input into the multimodal model, which combines visual information with semantic information to generate prompt words closely related to these visual elements. The generated prompt words can be directly used for subsequent image generation or other application scenarios.

[0074] In another specific implementation, the system can adjust the cropping of the image area according to specific business needs or contexts. For example, the system can automatically adjust the size or shape of the cropped area to ensure that the cropped area contains more relevant background information, so that the generated prompt words are more comprehensive and accurate. This method further optimizes the generation process of prompt words so that they better reflect the context of the target visual element in the overall image.

[0075] In other specific embodiments, the system can combine multiple target non-text visual elements and their corresponding position information, crop multiple image regions, and input these regions into the multimodal model at the same time. The multimodal model will process these different image regions comprehensively and generate one or more prompt words associated with these visual elements. This method can handle multiple related elements in complex scenes, and the generated prompt words are more diverse and have higher semantic relevance, which is suitable for multi-element image processing in complex scenes.

[0076] Through the above steps, the image area related to the target non-text visual element can be accurately cropped from the text-free prompt image, and highly relevant prompt words can be generated through the multimodal model. The accuracy and semantic consistency of prompt word generation are improved, ensuring that the generated prompt words can accurately reflect the characteristics of the target visual element and its context in the image. Not only does it enhance the flexibility of prompt word generation, it can also adapt to a variety of complex image scenes, improving the overall performance and practicality of the system in multimodal applications.

[0077] The present invention relates to the technical field of computer vision and natural language processing, and discloses a method for generating prompt words based on an image. By obtaining non-text visual elements and text elements and their position information from a prompt image, these elements are screened and processed to remove text content in the image and generate a prompt image without text. Based on the target non-text visual elements, the corresponding image area is cropped from the prompt image without text, and the area is input into a multimodal model to generate prompt words related to these visual elements. The present invention generates a text-free image by automatically identifying non-text visual elements and text elements in the image, thereby avoiding text interference. Prompt words that are highly related to the image content are generated, the accuracy and automation of prompt word generation are achieved, and the processing efficiency and prompt word quality are improved. The complexity of users writing prompt words is effectively reduced, so that users can generate high-quality and semantically accurate prompt words through simple image interaction.

[0078] In one embodiment, in the above S40, after inputting the target image area into the multimodal model to obtain the prompt words based on the non-text visual elements, the method further includes:

[0079] S501, inputting the image without text prompt into a multimodal embedding model to obtain an image embedding representation;

[0080] S502, inputting the prompt words based on the non-text visual elements into a multimodal embedding model to obtain an embedding representation of each target non-text visual element;

[0081] S503, calculating the cosine distance between the embedding representation of each target non-text visual element and the image embedding representation, and selecting a target prompt word from the prompt words based on the non-text visual elements based on the calculation result of the cosine distance.

[0082] In this embodiment, the textless prompt image and the generated prompt word are respectively input into the multimodal embedding model to obtain their embedded representations in the same semantic space. These embedded representations are high-dimensional vectors that can capture the semantic features of images and texts. On this basis, the similarity between the prompt word and the image embedding representation is evaluated by calculating the cosine distance between them. The smaller the cosine distance, the stronger the semantic association between the two. Therefore, the system selects the prompt word that best matches the image features by comparing these distances to ensure that the prompt word can accurately express the main visual elements and their context in the image.

[0083] In one specific implementation, the system first inputs a text-free prompt image into a multimodal embedding model to generate a global embedding representation of the image. This embedding representation can capture the overall features of the image as a comparison benchmark. Subsequently, the system inputs each prompt word generated based on non-text visual elements into the same multimodal embedding model to generate an embedding representation of each prompt word. By calculating the cosine distance between the embedding representation of each prompt word and the image embedding representation, the system can quantify the similarity between the prompt word and the overall features of the image. Based on the calculation results of these distances, the system selects the prompt word that is closest to the image features as the final target prompt word.

[0084] In another specific embodiment, after calculating the cosine distance, the system not only selects the prompt word closest to the image embedding representation, but also performs weighted processing on the prompt word. For example, the system can perform weighted adjustment on the calculated distance according to the importance of the prompt word or the specific relevance to the image content, so as to more accurately reflect the pros and cons of the prompt word. This method makes the selection of prompt words more flexible and can adapt to different application scenarios.

[0085] In other specific embodiments, the system can optimize the embedding representation of the prompt word in combination with the user's historical selection data. When selecting the target prompt word, the system not only considers the cosine distance between the prompt word and the image embedding representation, but also combines the user's historical selection preferences to further optimize the prompt word selection process. This approach can improve the personalization and accuracy of the prompt word, so that the generated prompt word is more in line with the user's specific needs.

[0086] Through the above steps, this embodiment can effectively evaluate and select the prompt words that best match the image features based on the multimodal embedding model. Not only is the accuracy of prompt word selection improved, but also the correlation and semantic consistency between the prompt words and the image content are enhanced through the calculation and weighted processing of the cosine distance. In addition, the optimization method combined with user historical data can further improve the personalization and applicability of the prompt words, ensuring that the final generated prompt words are more effective and meet user needs in practical applications.

[0087] In one embodiment, in the above S503, selecting a target prompt word from the prompt words based on the non-text visual element based on the calculation result of the cosine distance includes:

[0088] S5031, selecting a first preset number of preferred prompt words having the smallest calculation results of cosine distance from the prompt words based on the non-text visual element;

[0089] S5032, randomly selecting a second preset number of secondary selected prompt words from the remaining prompt words based on non-text visual elements except the first prompt word;

[0090] S5033: Combining the first choice prompt word with the second choice prompt word to obtain the target prompt word.

[0091] In this embodiment, after generating multiple prompt words based on non-textual visual elements, the system evaluates their similarity by calculating the cosine distance between the embedding representation of these prompt words and the image embedding representation. Then, the system selects the optimal target prompt word from the prompt words based on the similarity and the random selection strategy. This process comprehensively considers the semantic relevance and diversity of the prompt words, and by giving priority to the most relevant prompt words and combining some random selection, it ensures that the prompt words finally outputted are both accurate and retain a certain degree of creativity and flexibility.

[0092] In one specific implementation, the system can adopt an optimized strategy, firstly, based on a preset algorithm and weight model, calculate the cosine distance between each prompt word and the image embedding representation, and select the top N prompt words with the smallest distance as the preferred prompt words. These preferred prompt words represent the options most relevant to the image content, ensuring the semantic accuracy of the prompt words. This implementation is suitable for scenarios that require high-precision matching.

[0093] In another specific implementation, the system can introduce a machine learning model to assist in the selection process of the preferred prompt word. After calculating the cosine distance, the model can also consider the contextual information of the image content or the user's historical selection preferences to further optimize the selection of the preferred prompt word. In the selection of the secondary prompt word, the system can use an algorithm based on random sampling to ensure the diversity and flexibility of the prompt word selection, thereby generating more creative target prompt words.

[0094] In other specific implementations, the system can generate the final target prompt word by combining the preferred prompt word with the secondary prompt word through deep learning technology. This process can be achieved through semantic splicing, reordering, or context-based prompt word generation algorithm. This method not only ensures the semantic relevance of the prompt word, but also enriches the generated content by combining prompt words from different sources, and is suitable for complex content generation scenarios, such as advertising copy, creative design, etc.

[0095] Through the above steps, this embodiment can achieve a balance between accuracy and diversity, and the generated target prompt words are not only highly relevant to the image content, but also have certain flexibility and creativity. By combining similarity calculation and random selection strategy, the generated prompt words are prevented from being too single or lacking in novelty, while ensuring the quality and semantic consistency of the prompt words. This capability has significant application value in the fields of image generation, content creation, and automated marketing, and can effectively improve the diversity and applicability of the generated results.

[0096] In one embodiment, after the above S5033, the method further includes:

[0097] S601, using a pre-trained semantic analysis model, semantically matching and checking the content of each target prompt word, and adjusting the content of each target prompt word according to the semantic analysis result;

[0098] S602, obtaining the user's historical selection data, identifying the user's prompt word selection preference, and assigning a corresponding weight to each adjusted target prompt word;

[0099] S603: Score each target prompt word according to a preset quality standard and the weight of each target prompt word, and recommend the target prompt word with the highest score to the user.

[0100] In the present embodiment, first, by using the pre-trained semantic analysis model, the generated target prompt words are semantically matched and checked to ensure that the prompt words are semantically consistent with the target content. If semantic mismatch or logical error is found, the system will make adjustments based on the semantic analysis results. Next, the system uses the user's historical selection data to identify the user's prompt word selection preferences, and converts these preferences into weights, which are applied to the adjusted prompt words. Finally, the system scores the prompt words based on the preset quality standards and the weights of the prompt words, and recommends the prompt words with the highest scores to the user. This process combines semantic analysis, personalized recommendation and quality assessment to ensure that the prompt words recommended to the user are both accurate and in line with user preferences.

[0101] The semantic analysis model analyzes the semantic features of each cue word and matches it with the semantic features of the image content. This process involves converting the cue word into a semantic vector and comparing it with the semantic vector of the image content. If the model detects that the cue word is inconsistent with the semantics of the image content, such as some cue words cannot accurately describe an element in the image, the system will mark these cue words as objects that need to be adjusted.

[0102] For the marked prompt words, the system can automatically adjust their content based on the suggestions provided by the semantic analysis model. For example, if a word in the prompt word is inaccurate in the context, the system will replace it with a more appropriate word. Adjustments can also include optimizing the word order and grammatical structure of the prompt word to ensure that the prompt word is more semantically coherent and more in line with the overall context of the image.

[0103] The system will collect the user's historical selection data in similar scenarios. This data may include the type, style, and expression of the prompt words that the user has selected in the past.

[0104] Through data mining or machine learning algorithms, the system can identify user selection patterns, such as whether the user prefers to use concise descriptions, whether the user prefers to express in a certain tone, etc.

[0105] Based on the analysis results, the system assigns a weight to each target prompt word, which reflects the priority of the prompt word relative to the user's preference.

[0106] The weight allocation can be based on a variety of factors, such as the frequency of the user's historical selections, the degree of semantic matching of the prompt words, and the user's specific needs in the current scenario.

[0107] Example:

[0108] If a user has selected "concise and direct" prompts many times in the past image generation tasks, the system may give such prompts a higher weight. On the contrary, if the style or content of a prompt is obviously inconsistent with the user's historical preferences, the system may give it a lower weight. Ultimately, these weights will affect the ranking and recommendation of prompts.

[0109] In one specific implementation, the system can enhance the accuracy of semantic matching through an improved semantic analysis model. For example, the semantic analysis model can combine contextual semantics and cross-modal information to ensure that the prompt word is not only consistent with the image content in terms of meaning, but also semantically optimized according to the overall context of the image. In this way, the semantic accuracy and relevance of the prompt word are further improved.

[0110] In another specific implementation, when analyzing the user's historical selection data, the system can use a clustering algorithm to perform pattern recognition on the user's selection data and generate personalized prompt word weights accordingly. This method can more accurately reflect the user's preferences and optimize the final selection of prompt words through these preference weights, thereby providing a more customized user experience.

[0111] In other specific implementations, the system can introduce more quality criteria in the scoring stage, such as considering the diversity, innovation, and complementarity of the prompt words with other prompt words. This can not only improve the overall quality of the prompt words, but also provide more creative choices when recommending them to users, and enhance the effect of the prompt words in practical applications.

[0112] Through the above steps, this embodiment can ensure that the prompt words finally outputted are more semantically accurate, more consistent with the user's preferences, and meet high quality standards through various technical means during the process of generating the target prompt words.

[0113] In one embodiment, the above S30 includes:

[0114] S301, determining a text area to be removed in the prompt image based on the text element and the corresponding position information;

[0115] S302, performing transparent processing on the text area to be removed, removing the text elements, and generating a blank area;

[0116] S303, calculating the pixel average value within a preset adjacent range of the removed text element, and using the pixel average value to fill the blank area generated after the text element is removed, to generate the image without text prompt.

[0117] In this embodiment, first, the system will identify all text elements in the prompt image through a text detection model (such as OCR, Optical Character Recognition), and extract the location information of these elements in the image. OCR technology can scan every part of the image, identify possible text areas, and return the coordinate information of these areas.

[0118] Based on the detected text elements and their position information, the system determines the text area that needs to be removed. Specifically, this involves analyzing the position, size and shape of the text in the image to accurately define the bounding box of each text block. This step is very critical because it is necessary to ensure that the removed area exactly matches the original text area to avoid accidentally deleting other important visual information.

[0119] The core idea of ​​transparent processing is to completely remove the text area in the image without filling it immediately. After removal, the remaining area is "transparent", which means that the area does not show any color or content, but exists as a blank area without data. In image processing, transparent processing is usually achieved by using a "transparency channel" (such as the A channel in an RGBA image). The A channel represents the transparency of each pixel, so when processing, the pixels in the area can be made completely transparent by setting the A channel to 0. Through a text recognition algorithm (such as OCR), determine the text area that needs to be removed and obtain its precise location information. Set the recognized text area to transparent, and make the pixels in the area completely transparent by setting the A channel value to 0. This means that the text area disappears visually, but the information of its physical location is retained.

[0120] After the transparency processing, the system will analyze the generated blank area, calculate the average of the surrounding pixels, and use the average to fill the blank area. In this way, the originally transparent area will be filled with a color or texture similar to the surrounding area, ensuring that the image does not visually appear to have obvious missing or unnatural filling traces.

[0121] Pixel mean filling is a commonly used technique in image restoration. It calculates the average pixel value of adjacent areas to generate a fill that is consistent with the background. This method works particularly well in images with simple backgrounds and uniform colors. In addition, it can be used to maintain image consistency and coherence, avoiding obvious restoration marks in the filled area.

[0122] In addition, once the text area to be removed is determined, the system can also directly perform the overwriting operation. There are many ways to overwrite. The simplest way is to directly cover the area with a fixed color block (usually the background color) to completely remove the text element. This method is suitable for images with simple backgrounds and uniform colors.

[0123] For images with complex backgrounds, simple overlays may cause obvious visual artifacts. Therefore, more complex algorithms can be used, such as overlaying the text area with a color or texture similar to the surrounding pixels. This method ensures that the overlay area blends naturally with the surrounding image parts and reduces the traces of repair.

[0124] After removing the text, the system will calculate the average value of the pixels in the preset adjacent range of the removed text area. This range usually refers to the pixels within a certain distance around the text area, such as 3x3, 5x5, etc. By calculating the average color value of these adjacent pixels, a fill content that is consistent with the surrounding background color can be generated.

[0125] The calculated mean is used to fill the removed text area. This process can be simply understood as "copying" the surrounding colors and "pasting" them into the blank area, so that the patched area visually looks naturally integrated with the surrounding image parts. This method is particularly suitable for images with simple backgrounds and smooth color transitions.

[0126] For complex backgrounds, the system can use texture synthesis methods or deep learning-based image restoration techniques. These techniques generate a fill content that matches the overall background texture by analyzing the surrounding image texture. This method not only considers color, but also texture features in the image, making the repaired area more visually natural and coherent. For example, generative adversarial networks (GANs) can be used for more complex image restoration tasks. GAN models can learn how to fill in different types of image content through training, especially when the image background is complex. GAN models can generate more realistic fill content by learning the surrounding pixel patterns.

[0127] In one specific implementation, the system first uses OCR (optical character recognition) technology to identify all text elements from the prompt image. After identification, the system calculates the bounding box of the text area based on the position information of the text elements. These bounding boxes are used to identify the areas that need to be removed. For these areas, the system removes them through a simple color covering technique. For example, in an image with a single background color, the system can directly fill these areas with the background color to generate a prompt image without text.

[0128] In another specific embodiment, the system can combine the background information of the image to perform more complex overlay operations when removing text. For images with complex backgrounds, the system can generate fill content that is consistent with the surrounding environment by analyzing the pixel color and texture information around the text area. For example, using a clone tool or smart fill algorithm, the system can copy the texture features of the adjacent area and apply it to the blank area after removal, ensuring that the generated image is visually consistent and seamless.

[0129] In other specific embodiments, the system can use a generative adversarial network (GAN) or other deep learning technology to fill in the area where the text is removed with high precision. The GAN model learns the texture and structure in the image to generate filling content that is completely consistent with the image background. For images that require high-precision restoration, such as complex scenes or high-resolution images, GAN can significantly improve the filling effect, making the generated text-free image more visually natural and almost invisible.

[0130] Through the above steps, this embodiment can successfully remove text elements in the image and generate a text-free prompt image while maintaining the overall visual consistency of the image. This process not only improves the accuracy and automation of image processing, but also ensures that the image is not disturbed in subsequent processing, and is suitable for a wide range of image editing and generation application scenarios.

[0131] In one embodiment, in the above S30, after inputting the target image area into the multimodal model to obtain the prompt words based on the non-text visual elements, the method further includes:

[0132] S701, performing a grammatical and logical consistency check on the prompt word based on the non-text visual element based on a preset semantic rule, and if a grammatical and / or logical error is found, correcting the prompt word based on the non-text visual element;

[0133] S702: Generate multiple versions of candidate prompt words according to the revised prompt words based on the non-text visual element, and select a final prompt word based on the non-text visual element from the multiple candidate prompt words according to a preset selection criterion.

[0134] In this embodiment, after generating the prompt words based on non-text visual elements, the system first performs a grammatical check on the prompt words using preset semantic rules and natural language processing models. This check includes identifying common grammatical errors, such as subject-verb inconsistency, incorrect tense usage, singular-plural mismatch, etc. This step ensures that the generated prompt words are grammatically correct and avoids affecting the readability and accuracy of the prompt words due to grammatical problems.

[0135] In addition to grammar checking, the system also analyzes the logical consistency of the prompt word. This includes checking the logical coherence of the sentences before and after the prompt word, the consistency of the content, etc. If the system finds that the prompt word is logically inconsistent, such as the description does not match the image content, the system will automatically correct it. This step ensures that the prompt word is not only grammatically correct, but also logically coherent, so that it can accurately convey the semantics of the image.

[0136] During the grammar and logic check, if any errors or inconsistencies are found, the system will automatically correct the prompt word according to the semantic rules. For example, the system may correct the prompt word by adjusting the sentence structure, replacing inappropriate words, or reordering the sentence. This step ensures that the generated prompt word is more accurate and consistent in semantics and logic.

[0137] Based on the revised prompt words, the system will generate multiple versions of candidate prompt words. These candidate versions may differ in wording, tone, sentence structure, etc. to adapt to different application scenarios. The purpose of generating multiple versions is to provide diverse choices and ensure that the most appropriate prompt words can be found in different contexts.

[0138] The system will evaluate and select multiple candidate prompt words based on a series of preset criteria. The criteria may include grammatical correctness, logical consistency, relevance to the image content, simplicity of expression, etc. The system scores each candidate prompt word and finally selects the prompt word with the highest score as the final output. This ensures that the final selected prompt word is the best in quality and applicability.

[0139] In one specific implementation, the system first inputs the target image area into the multimodal model to generate prompt words based on non-text visual elements. These prompt words are then sent to the grammar and logic consistency check module, which detects grammatical and logical problems in the prompt words based on preset semantic rules. After discovering problems, the system automatically corrects the prompt words to ensure their accuracy. Next, the system generates multiple different versions of prompt words for use in different application scenarios. These versions may include changes in tone, wording or structure. Finally, the system evaluates these candidate prompt words according to a set of preset selection criteria and finally selects the optimal prompt word as the final output.

[0140] In another specific implementation, after generating the prompt words and performing preliminary grammar and logic checks through the semantic analysis model, the system will generate multiple versions of candidate prompt words based on the user's historical selection data and context information. For example, the system can generate prompt words of different styles considering factors such as the user's language habits and industry background. Among the multiple candidate prompt words generated, the system will apply preset selection criteria, which may include the simplicity, professionalism, and relevance of the prompt words to the image content. The system selects the prompt word with the highest score as the prompt word that is finally recommended to the user through a weighted scoring method.

[0141] In other specific embodiments, the system will not only check and correct the prompt words based on semantic rules, but also generate multiple candidate prompt words in combination with the specific requirements of the image content. The system may generate a concise version for real-time feedback scenarios, or a detailed version for reporting and analysis. These candidate prompt words will be sorted and screened according to preset standards, such as accuracy, relevance, readability, etc. The system will provide the prompt words that best meet the user's needs when making the final recommendation, while allowing the user to make further modifications and selections when necessary to ensure maximum personalization and practicality.

[0142] Through the above steps, this embodiment can fully consider the correctness of grammar and logic in the whole process of generating prompt words, ensuring that the prompt words finally generated are semantically accurate. At the same time, the system provides multiple candidate versions and screens them according to specific standards, so that the prompt words finally output not only meet the semantic standards, but also meet different application requirements.

[0143] In one embodiment, the above S20 includes:

[0144] S201, determining whether an element retention instruction is detected;

[0145] S202, if yes, filter out the target non-text visual element from the non-text visual elements according to the element retention instruction;

[0146] S203: If not, select a third preset number of target non-text visual elements from the non-text visual elements.

[0147] In this embodiment, the element retention instruction is a specific instruction issued by the user through the system interface or other input methods (such as voice instructions) to indicate which non-text visual elements the system should retain or prioritize. When processing an image, the system first needs to determine whether such an instruction is detected. The core of this step is the coordination between user interaction and system automated processing to ensure that the user's intentions can be accurately reflected in image processing.

[0148] The system monitors user input signals in real time to detect whether explicit retention instructions are issued. If instructions are detected, the system will record them and give priority to user-specified elements in subsequent processing. This feature allows users to customize and control the system's image processing process, improving the system's flexibility and adaptability.

[0149] After detecting the element retention instructions, the system needs to filter out the target elements from all recognized non-text visual elements based on these instructions. The key to this step is how the system accurately matches the user instructions with the elements in the actual image to ensure that the user-specified elements are correctly recognized and processed.

[0150] The system selects target elements that match the instructions by matching keywords or specific parameters (such as element type, location, etc.) in the user's instructions with the element features in the image. This may include complex image analysis and element recognition algorithms to ensure that the selected elements are consistent with the user's needs. The system then marks these target elements as objects that need to be processed first and enters the next processing flow.

[0151] In the absence of a retention instruction, the system needs to autonomously decide which non-text visual elements should be further processed as target elements. At this point, the system automatically selects a certain number of target elements from all identified elements based on preset rules or algorithms. This process relies on the system's automated processing capabilities and intelligent decision-making mechanisms.

[0152] The system selects a predetermined number of target elements from all non-text visual elements using preset screening criteria (such as element prominence, size, distance from the center of the image, etc.). These criteria are usually defined during system design and can be adjusted according to specific application scenarios. For example, the system can select the top N most prominent elements in an image as target elements to ensure that these elements are prioritized in subsequent processing. Through this automated screening, the system can ensure the efficiency and effectiveness of image processing in the absence of user instructions.

[0153] In one specific implementation, the system first identifies and extracts non-text visual elements in the image, such as objects, scenes, or other visual features. The system monitors whether there are any element retention instructions issued by the user, such as the user selecting specific elements through the interface or specifying the elements to be retained through voice instructions. If such retention instructions are detected, the system will accurately filter out the target non-text visual elements specified by the user based on these instructions to ensure that these elements are prioritized or retained. This method is suitable for scenarios where users need to filter key visual elements manually or through specific instructions, providing higher interactivity and control.

[0154] In another specific embodiment, the system uses a preset automatic screening mechanism for screening when no element retention instruction is detected. The system can automatically select the top several non-text visual elements with the highest scores as target elements based on the analysis results of the image content. This score may be based on the prominence, size, color or other characteristics of the element. For example, the system can select the top five most prominent elements in the image as target non-text visual elements for subsequent processing. This method is suitable for scenarios where the user does not participate in the screening process and wants the system to automatically select important elements.

[0155] In other specific embodiments, the system can dynamically adjust the third preset number of target non-text visual elements in combination with the user's historical data or the contextual information of the current task without explicit retention instructions. For example, in certain specific scenarios, if the user's historical selection data indicates that they prefer a specific type of element, the system can automatically adjust the preset number or type to better match the user's needs. This method improves the adaptability and user experience of the system through an intelligent dynamic adjustment mechanism, and is suitable for a variety of complex application scenarios.

[0156] Through the above steps, this embodiment can flexibly and efficiently filter out target elements from non-text visual elements, whether through user instructions or automated filtering mechanisms. This not only improves the accuracy of the filtering process, but also increases the intelligence and adaptability of the system, and can better meet user needs and diverse application scenarios.

[0157] The present invention also provides a device for generating prompt words based on an image, referring to Figure 2 , Figure 2 The following is a functional module diagram of a preferred embodiment of an image-based prompt word generation device of the present invention. The image-based prompt word generation device comprises:

[0158] An image recognition module 10 is used to obtain a prompt image and recognize non-text visual elements, text elements and corresponding position information of the prompt image;

[0159] A target screening module 20, used to screen out target non-text visual elements from the non-text visual elements;

[0160] A text removal module 30, used to remove the text in the prompt image based on the text element and the corresponding position information to generate a text-free prompt image;

[0161] The prompt word generation module 40 is used to crop a target image area from the text-free prompt image based on the target non-text visual element and the corresponding position information, input the target image area into the multimodal model, and obtain a prompt word based on the non-text visual element.

[0162] The specific implementation of the image-based prompt word generation device of the present invention is basically the same as the above-mentioned various embodiments of the image-based prompt word generation method, and will not be repeated here.

[0163] The present invention also provides a computer device, such as Figure 3As shown, the computer device may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0164] Those skilled in the art will understand that Figure 3 The hardware structure of the computer device shown in the figure does not constitute a limitation on the computer device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0165] like Figure 3 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and an image-based prompt word generation program. The operating system is a program for managing and controlling computer devices and software resources, and supports the operation of the network communication module, the user interface module, the image-based prompt word generation program, and other programs or software; the network communication module is used to manage and control the network interface 1004; and the user interface module is used to manage and control the user interface 1003.

[0166] exist Figure 3 In the computer device hardware structure shown, the network interface 1004 is mainly used to connect to the background server and communicate data with the background server; the user interface 1003 is mainly used to connect to the client and communicate data with the client; the processor 1001 can call the image-based prompt word generation program stored in the memory 1005 and perform the same operation as the image-based prompt word generation method.

[0167] The specific implementation of the computer device of the present invention is basically the same as the above-mentioned embodiments of the image-based prompt word generation method, and will not be repeated here.

[0168] In addition, an embodiment of the present invention further provides a computer storage medium on which an image-based prompt word generation program is stored. When the image-based prompt word generation program is executed by a processor, the steps of the image-based prompt word generation method described above are implemented.

[0169] The specific implementation of the computer storage medium of the present invention is basically the same as the above-mentioned embodiments of the image-based prompt word generation method, and will not be repeated here.

[0170] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims. All equivalent structures or equivalent process changes made using the contents of the specification and drawings of the present invention, or directly or indirectly used in other related technical fields, are protected by the present invention.

[0171] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to the multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of boxes in the block diagram and / or the flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0172] In addition, each functional module in each embodiment of the present application can be integrated together to form an independent part, or each module can exist separately, or two or more modules can be integrated to form an independent part. If the function is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, RandomAccess Memory), a disk or an optical disk.

[0173] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0174] The above are only various implementations of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0175] It should be noted that if software tools or components other than those of our company appear in the embodiments of the present application, they are only used for illustrative purposes and do not represent actual use.

Claims

1. A method for generating prompt words based on an image, characterized in that: The following steps are involved: Acquire a prompt image, and identify non-text visual elements, text elements, and corresponding position information of the prompt image; Filtering out target non-text visual elements from the non-text visual elements; Based on the text element and the corresponding position information, remove the text in the prompt image to generate a prompt image without text; A target image area is cropped from the image without text prompt based on the target non-text visual element and the corresponding position information, and the target image area is input into a multimodal model to obtain a prompt word based on the non-text visual element.

2. The image-based prompt word generation method according to claim 1, characterized in that: After inputting the target image area into the multimodal model to obtain a prompt word based on a non-text visual element, the method further includes: Inputting the image without text prompt into a multimodal embedding model to obtain an image embedding representation; Inputting the prompt words based on the non-text visual elements into a multimodal embedding model to obtain an embedding representation of each target non-text visual element; A cosine distance between an embedding representation of each target non-text visual element and an embedding representation of the image is calculated, and a target prompt word is selected from the prompt words based on the non-text visual element based on the calculation result of the cosine distance.

3. The image-based prompt word generation method according to claim 2, characterized in that: The step of selecting a target prompt word from the prompt words based on the non-text visual element based on the calculation result of the cosine distance includes: Selecting a first preset number of preferred prompt words with the smallest calculation results of cosine distance from the prompt words based on the non-text visual element; Randomly select a second preset number of secondary selected prompt words from the remaining prompt words based on non-textual visual elements except the first prompt word; The target prompt word is obtained by combining the first prompt word and the second prompt word.

4. The image-based prompt word generation method as claimed in claim 3, characterized in that: After combining the first choice prompt word with the second choice prompt word to obtain the target prompt word, the method further includes: Use the pre-trained semantic analysis model to semantically match and check the content of each target prompt word, and adjust the content of each target prompt word according to the semantic analysis results; Obtain the user's historical selection data, identify the user's prompt word selection preference, and assign corresponding weights to each adjusted target prompt word; Each target prompt word is scored according to the preset quality standard and the weight of each target prompt word, and the target prompt word with the highest score is recommended to the user.

5. The image-based prompt word generation method according to claim 1, characterized in that: The removing the text in the prompt image based on the text element and the corresponding position information to generate a prompt image without text includes: Based on the text element and the corresponding position information, determining a text area in the prompt image that needs to be removed; Performing transparent processing on the text area to be removed, removing the text element, and generating a blank area; The pixel average value within a preset adjacent range of the removed text element is calculated, and the blank area generated after the text element is removed is filled with the pixel average value to generate the image without text prompt.

6. The image-based prompt word generation method according to claim 1, characterized in that: After inputting the target image area into the multimodal model to obtain a prompt word based on a non-text visual element, the method further includes: Performing a grammatical and logical consistency check on the prompt words based on the non-textual visual elements based on preset semantic rules, and if grammatical and / or logical errors are found, correcting the prompt words based on the non-textual visual elements; A plurality of versions of candidate prompt words are generated according to the revised prompt words based on the non-textual visual element, and a final prompt word based on the non-textual visual element is selected from the plurality of candidate prompt words according to a preset selection criterion.

7. The image-based prompt word generation method according to claim 1, characterized in that: The step of selecting a target non-text visual element from the non-text visual elements includes: Determine whether an element retention instruction is detected; If so, filtering out a target non-text visual element from the non-text visual elements according to the element retention instruction; If not, a third preset number of target non-text visual elements are selected from the non-text visual elements.

8. A prompt word generation device based on an image, characterized in that: The image-based prompt word generating device comprises: An image recognition module, used to obtain a prompt image, and recognize non-text visual elements, text elements, and corresponding position information of the prompt image; A target screening module, used for screening out target non-text visual elements from the non-text visual elements; A text removal module, used to remove the text in the prompt image based on the text element and the corresponding position information, and generate a text-free prompt image; The prompt word generation module is used to crop a target image area from the text-free prompt image based on the target non-text visual element and the corresponding position information, input the target image area into the multimodal model, and obtain a prompt word based on the non-text visual element.

9. A computer device, characterized in that: The computer device includes a memory, a processor, and an image-based prompt word generation program stored in the memory and executable on the processor. When the image-based prompt word generation program is executed by the processor, the steps of the image-based prompt word generation method as described in any one of claims 1 to 7 are implemented.

10. A computer storage medium, characterized in that: The storage medium stores an image-based prompt word generation program, and when the image-based prompt word generation program is executed by the processor, the steps of the image-based prompt word generation method according to any one of claims 1 to 7 are implemented.

Images