Package illustration generation method and device, electronic equipment and readable storage medium

Abstract

The invention discloses a package illustration picture generation method and device, electronic equipment and a readable storage medium, relates to the technical field of image processing, and can automatically analyze and determine each sub-commodity included in a package and select an independent illustration picture, the independent illustration pictures are combined by adopting a style layout template matched with the package commodity, and the image processing efficiency is improved. According to the method, the complete package illustration is generated, a merchant does not need to illustrate the package by himself / herself, the consistency of illustration styles and the controllability of quality are ensured while the illustration process of the package is simplified, and the overall visual effect of the package illustration is improved. The method comprises the following steps: determining package text information of a to-be-generated illustration, and analyzing the package text information to obtain a sub-commodity set corresponding to the package text information; based on the sub-commodity set, selecting a plurality of sub-commodity illustrations, and determining at least one style layout template matched with the package text information; and combining and arranging the plurality of sub-product illustrations by referring to at least one style layout template to obtain at least one package illustration and outputting the package illustration.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device, electronic device and readable storage medium for generating meal set images. Background Art

[0002] With the rapid development of Internet technology, especially the rise of artificial intelligence content generation technology, all walks of life are undergoing digital transformation. In the local catering industry, this change is particularly significant, which not only improves operational efficiency, but also greatly enriches consumers' selection experience. Among them, package products, as an important part of meeting users' rigid needs, account for a significant proportion of transaction contributions on e-commerce platforms. As the first visual element to attract consumers' attention, the quality of package pictures directly affects the click-through rate and conversion rate of products.

[0003] In the related art, when making pictures of set meals, it is usually necessary to rely on professional photographers and design teams to shoot the dishes before making them, or the merchants can shoot and make them by themselves. The applicants realize that in this process, a lot of manpower, material resources and time costs are consumed, the process is cumbersome and inefficient, and it is difficult to ensure the consistency of the picture style and the controllability of the quality due to the professionalism of the tools, the level of visual design and the lack of unified production standards. The overall visual effect of the set meal pictures is not high. Summary of the invention

[0004] In view of this, the present application provides a method, device, electronic device and readable storage medium for generating meal set pictures. The main purpose is to solve the problem that the current process requires a lot of manpower, material resources and time costs, the process is cumbersome and inefficient, and is limited by the professionalism of the tools, the level of visual design and the lack of unified production standards. It is difficult to ensure the consistency of the picture style and the controllability of the quality, and the overall visual effect of the meal set pictures is not high.

[0005] According to the first aspect of the present application, a method for generating a set meal image is provided, the method comprising:

[0006] Determine the text information of the package for which the picture is to be generated, parse the text information of the package, and obtain a sub-product set corresponding to the text information of the package;

[0007] Based on the sub-product set, multiple sub-product pictures are selected, and at least one style layout template matching the package text information is determined;

[0008] The plurality of sub-product pictures are combined and laid out with reference to the at least one style layout template to obtain and output at least one set meal picture.

[0009] According to a second aspect of the present application, a set meal image generation device is provided, the device comprising:

[0010] A parsing module, used to determine the text information of the package for which the accompanying image is to be generated, parse the text information of the package, and obtain a sub-product set corresponding to the text information of the package;

[0011] A determination module, configured to select a plurality of sub-product pictures based on the sub-product set, and determine at least one style layout template matching the package text information;

[0012] The combination module is used to combine and layout the multiple sub-product pictures with reference to the at least one style layout template, obtain at least one set meal picture and output it.

[0013] According to a third aspect of the present application, an electronic device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the methods described in the first aspect when executing the computer program.

[0014] According to a fourth aspect of the present application, a readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the methods described in the first aspect are implemented.

[0015] By means of the above technical scheme, the present application provides a method, device, electronic device and readable storage medium for generating meal set pictures. The present application determines the meal set text information for which pictures are to be generated, parses the meal set text information, obtains a sub-product set corresponding to the meal set text information, selects multiple sub-product pictures based on the sub-product set, and determines at least one style layout template matching the meal set text information, combines and lays out the multiple sub-product pictures with reference to the at least one style layout template, obtains at least one meal set picture and outputs it, and can automatically parse and determine the names of the sub-products included in the meal set text information and select an independent picture for each sub-product name, and uses a style layout template matching the meal set text information to combine the independent pictures to generate a complete meal set picture, without the need for merchants to arrange pictures for the meal sets by themselves, and while simplifying the picture arrangement process for the meal set and improving the picture arrangement efficiency, uses the style layout template to ensure the consistency of the picture arrangement style and the controllability of the quality, thereby improving the overall visual effect of the meal set picture.

[0016] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0018] Figure 1 A schematic diagram of a method for generating a meal set with a diagram provided in an embodiment of the present application is shown;

[0019] Figure 2A A schematic diagram of another method for generating meal set images provided in an embodiment of the present application is shown;

[0020] Figure 2B A schematic diagram of package splitting provided in an embodiment of the present application is shown;

[0021] Figure 2C A schematic diagram of entity tag recognition provided by an embodiment of the present application is shown;

[0022] Figure 2D A schematic diagram of a sub-product set generation process provided in an embodiment of the present application is shown;

[0023] Figure 2E A schematic diagram of a preset style layout template generation process provided in an embodiment of the present application is shown;

[0024] Figure 2F A schematic diagram of a layout intelligent adjustment process provided by an embodiment of the present application is shown;

[0025] Figure 2G A schematic diagram of a sub-product matching process provided in an embodiment of the present application is shown;

[0026] Figure 2H A schematic diagram showing the effect of intelligently adjusting the layout provided by an embodiment of the present application is shown;

[0027] Fig.2I A schematic diagram of a process flow of another method for generating a meal set with a diagram provided in an embodiment of the present application is shown;

[0028] Figure 3 A schematic diagram of the structure of a set meal diagram generating device provided in an embodiment of the present application is shown;

[0029] Figure 4 A schematic diagram of the device structure of a computer device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0030] The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0031] The present application embodiment provides a method for generating a meal set with a picture, such as Figure 1 As shown, the method includes:

[0032] 101. Determine the text information of the package for which the accompanying image is to be generated, parse the text information of the package, and obtain a sub-product set corresponding to the text information of the package.

[0033] The embodiment of the present application can be applied to a picture matching system, which can parse the text information of the set meal products provided by the merchant, identify the main products, side dishes, beverages and other sub-products included in the set meal products, and automatically generate set meal matching pictures for the set meal products for the merchant's reference and use. The picture matching system is suitable for various e-commerce platforms, catering companies, retail stores and other scenarios where set meal products need to be displayed. By generating high-quality product matching pictures, the consistency of the matching style and the controllability of the quality are guaranteed, and the overall visual effect of the set meal matching pictures is improved.

[0034] During the application process, the picture matching system will receive the product package specified by the merchant or the system, and use the text information associated with the product package name of the product package as the package text information to be generated for the picture matching, parse the package text information, extract the sub-product names of each sub-product included therein, and obtain the sub-product set corresponding to the package text information. Among them, the package text information is the text description information associated with the product combination for which the picture matching needs to be generated, such as a package containing a main dish, a side dish, and a beverage, which is also a product package, and the text description information of the package is the package text information; the sub-product set is generated based on the product information of each independent product that constitutes the package text information, such as the main dish, side dish, and beverage are respectively used as sub-products, and the sub-product names of these sub-products are extracted and the sub-product set is formed based on the information of these sub-products.

[0035] In this way, the picture matching system can accurately identify the composition of the package text information, provide basic data for subsequent picture matching generation, and obtain the sub-product set through analysis, so as to facilitate the matching of pictures for each sub-product and improve the picture matching quality.

[0036] 102. Based on the sub-product set, select multiple sub-product pictures, and determine at least one style layout template that matches the package text information.

[0037] In the embodiment of the present application, after the sub-product set is generated, the picture matching system will select multiple sub-product pictures based on the sub-product set, and determine at least one style layout template that matches the package text information according to the characteristics of the package text information. Among them, the sub-product picture is an independent product picture that matches each independent sub-product in the sub-product, which is used for the subsequent synthesis of the package picture; the style layout template is a template for combining and arranging the sub-product pictures, which defines the overall style and layout method of the picture, so as to ensure that the generated package picture conforms to the style and characteristics of the package text information.

[0038] In this way, by selecting illustrations that match the sub-products, the accuracy and attractiveness of the illustration content can be ensured. At the same time, by selecting style layout templates, a unified style and layout of the illustrations can be achieved, thereby improving the overall aesthetics.

[0039] 103. Referring to at least one style layout template, multiple sub-product pictures are combined and laid out to obtain and output at least one set meal picture.

[0040] In the embodiment of the present application, the picture matching system will refer to the selected at least one style layout template, combine and layout multiple sub-product pictures, obtain at least one package picture and output it for the merchant to view or use. Among them, the package picture is a complete picture generated based on the matched independent sub-product pictures, which shows all the sub-products included in the package text information. Therefore, through combination and layout, the picture matching system integrates multiple sub-product pictures into a complete package picture, which is convenient for users to intuitively understand the content of the package, without the need for merchants to make package pictures by themselves, simplifying the package picture matching process and improving picture matching efficiency.

[0041] The method provided in the embodiment of the present application determines the package text information for which the illustration is to be generated, parses the package text information, obtains a sub-product set corresponding to the package text information, selects multiple sub-product illustrations based on the sub-product set, and determines at least one style layout template matching the package text information, combines and lays out the multiple sub-product illustrations with reference to the at least one style layout template, obtains and outputs at least one package illustration, and can automatically parse and determine the names of the sub-products included in the package text information and select an independent illustration for each sub-product name, and combines the independent illustrations using a style layout template matching the package text information to generate a complete package illustration. Merchants do not need to prepare illustrations for the packages by themselves, and while simplifying the illustration process for the packages and improving the illustration efficiency, the style layout template is used to ensure the consistency of the illustration style and the controllability of the quality, thereby improving the overall visual effect of the package illustration.

[0042] Further, as a refinement and extension of the specific implementation of the above embodiment, in order to fully illustrate the specific implementation process of this embodiment, the embodiment of the present application provides another method for generating a meal set with a picture, such as Figure 2A As shown, the method includes:

[0043] 201. Determine the text information of the meal package for which the accompanying image is to be generated, parse the text information of the meal package, and obtain a sub-product set corresponding to the text information of the meal package.

[0044] The applicants recognize that package products, as important core supply assets, need to further improve the coverage and richness of packages, especially the low quality of package pictures, which need to be optimized. However, most package pictures are made by merchants themselves. Due to limited tools and visual level, the quality of package pictures is very low, and there are the following problems: the package sub-product pictures cannot search for products (small gallery), the content is not accurately expressed (inconsistent pictures and text); the product pictures are of poor quality and difficult to cut out; the perspectives of multiple product pictures are inconsistent, so the splicing causes visual confusion; the main and secondary dishes are not distinguished, the order of placement is wrong, and the size of objects is abnormal; the dishes cover each other, or there is an unreasonable intersection with the background image elements, which causes visual unacceptability.

[0045] Therefore, the embodiment of the present application proposes a method for matching pictures of set meals, which can automatically parse and determine the various sub-commodities included in the set meal and select independent pictures for each sub-commodity, and use a style layout template that matches the set meal product to combine the independent pictures to generate a complete set meal picture, without the need for merchants to match pictures of the set meal themselves. While simplifying the picture matching process of the set meal and improving the picture matching efficiency, the style layout template is used to ensure the consistency of the picture matching style and the controllability of the quality, thereby improving the overall visual effect of the set meal picture matching. The embodiment of the present application can be applied to a picture matching system, which can parse the set meal products provided by the merchant, identify the main products, side dishes, beverages and other sub-commodities included in the set meal products, and automatically generate a set meal picture for the set meal products for reference and use by the merchant. The picture matching system is suitable for various scenarios such as e-commerce platforms, catering companies, and retail stores that need to display set meal products. By generating high-quality product pictures, the consistency of the picture matching style and the controllability of the quality are ensured, thereby improving the overall visual effect of the set meal picture matching. Among them, in order to realize the analysis of the package products, determine the sub-products included in the package products and the attributes and characteristics of each sub-product, so as to facilitate the subsequent matching of independent product pictures for the sub-products, a product map model and a multimodal visual model are pre-constructed in the embodiment of the present application, and the package products are analyzed and split using the product map model and the multimodal visual model. The two models are introduced and their training process is described below:

[0046] 1. Product graph model.

[0047] The product graph model can be a knowledge graph model that can perform named entity recognition. Named Entity Recognition (NER) refers to identifying entities with specific meanings from natural language texts, such as names of people, places, and dishes, and classifying them. NER is crucial in the construction of knowledge graphs because it can help extract structured information from unstructured data, thereby enriching the content of the knowledge graph. The mapping system will match multiple sample product information, analyze the product attributes of each sample product information, and build a product graph model based on multiple sample product information and the product attributes of each sample product information. Among them, when analyzing product attributes, considering that there is information about package products in the sample product information, and an entity in the package text of the package product contains multiple characters and there is a problem of entity nesting, therefore, in the embodiment of the present application, an algorithm for extracting relationships and entities is used to analyze the product attributes of the sample product information. The specific process is as follows:

[0048] First, match the products on the platform where the merchant's store is located, use the product information of multiple matched products as multiple sample product information, and use the pre-trained deep language model to adaptively fine-tune each sample product information. Specifically, the lightweight Chinese pre-trained model ALBERT can be used to perform fine-tune on the package text to achieve adaptive fine-tuning of product information.

[0049] Next, a two-layer bidirectional long short-term memory network is used to encode each sample product information that has undergone adaptive fine-tuning, and the text feature information corresponding to each sample product information is obtained. The text feature information of each sample product information is input into the conditional random field for parsing, and at least one entity tag corresponding to each sample product information is obtained, and the at least one entity tag obtained is used as the product attribute of the corresponding sample product information. Specifically, the text can be encoded by a two-layer bidirectional LSTM, and the encoding result can be input into the CRF layer to extract the entity tag.

[0050] 2. Multimodal vision model.

[0051] The multimodal visual model can be a VLM (Vision-Language Model) visual model. The VLM visual model is a deep learning model that combines vision and language understanding. It improves the model's understanding and generation capabilities by fusing information from the two modalities. It is usually used to process and generate the relationship between images and texts, and is widely used in tasks such as image description, image question and answer, and visual search.

[0052] When training a multimodal visual model, multiple sample questions can be set, and an answer template corresponding to each sample question can be constructed. The initial visual model can be guided and trained using the multiple sample questions and the answer template corresponding to each sample question to obtain a multimodal visual model, wherein the multiple sample questions include text questions and image questions. For example, when applied in the catering industry, the following types of sample questions can be set:

[0053] Text questions: "Please describe the cooking method and taste of this dish." Such questions are designed to allow the model to learn to parse the cooking characteristics and sensory experience of food from the text description.

[0054] Image-based questions: "Based on this picture, identify and list all the ingredients." This type of question requires the model to accurately identify the various ingredients shown in the picture through image recognition technology, which is particularly important for automatically identifying ingredients and matching pictures.

[0055] Next, you can design a corresponding answer template for each of the above types of questions. For text questions, the answer template may include a detailed description of the food's cooking method, flavor, etc., while for image questions, the answer template may be a structured list of ingredients.

[0056] Using these sample questions and answer templates, the initial visual model can be trained in detail. Through continuous iteration and optimization, the initial visual model will gradually learn to establish accurate associations between text and images, thereby demonstrating excellent capabilities in processing multimodal information in the catering field.

[0057] Through the above process, the product graph model and the multimodal visual model are trained. In this way, when merchants have the need for picture matching, they can rely on the product graph model and the multimodal visual model to split the package text information, determine the sub-product names included therein, and accurately identify the attributes corresponding to the sub-product names. Among them, package splitting is to identify the sub-product names contained in the product package from the existing package text information of the product package, which is the basis of downstream applications, because only by fully and accurately understanding what the sub-products in the product package are, can the pictures be customized in a targeted manner. In order to meet the flexible customization needs of merchants, the picture matching system provides two structures: structured packages and text packages. The structured package only contains the single products sold in the store, and the sub-products can be obtained directly through the products. The text package is a package freely configured by the merchant, and it is necessary to use an algorithm to convert the unstructured package text information into structured sub-product information. The unstructured package text information contains the product name and product description of the package. Since the text has a hierarchical structure, it belongs to a reading comprehension problem. It is necessary not only to identify the sub-products from the package, but also to mine the relationship between the sub-products. Therefore, the process of generating a sub-product set can be divided into two parts, one is the recognition of named entities in the text, and the other is the extraction of relationships between entities. The embodiment of the present application extracts information from a product graph model and understands and detects a comprehensive operation of a multimodal visual model to extract the recommended standard dish names corresponding to the sub-products based on the product name and the sub-products associated with the store, accurately determines the type of the sub-product: main dish, side dish, drink, and infers the core attributes of the set meal such as the category to which the set meal belongs, thereby generating a sub-product set. The process of generating a sub-product set is described below, including the following steps 2011 to 2015:

[0058] 2011. In response to a request for generating a picture from a merchant, determine a product package name indicated by the request for generating a picture, and use text information associated with the product package name as package text information for generating a picture.

[0059] When a merchant has a need for illustrations, a request for illustration generation will be sent to the illustration system. The illustration generation request can be a request submitted by the merchant through a specific interface or screen, which is intended to generate visual display pictures for its product packages. In response to the merchant's illustration generation request, the illustration system first identifies and confirms the product package name clearly indicated in the illustration generation request, and uses the indicated product package as the target product package to generate illustrations. A product package refers to a collection of multiple products or service items sold by a merchant in combination, usually containing specific discount information or added value to meet the diverse needs of consumers.

[0060] After determining the target product package, the illustration system further obtains text information of the target product package as package text information. The package text information includes but is not limited to key data such as package name, included product list and description, price, discount details, validity period, applicable conditions, etc. The package text information is crucial to the accuracy and attractiveness of the subsequent illustration design.

[0061] Through the above process, not only is it ensured that the image generation work can accurately correspond to the actual needs of the merchants, avoiding resource waste and inefficiency caused by information mismatch, but also by comprehensively collecting package text information, it provides a rich and accurate material basis for subsequent image design, which helps to design high-quality images that are in line with the characteristics of the product and can attract potential consumers, thereby improving the overall visual effect of the package images.

[0062] 2012. The product graph model is used to extract entity semantic information from the package text information to obtain multiple sub-product names and the product entity information corresponding to each sub-product name.

[0063] When identifying the entity information of the product, the image matching system will use the product graph model to conduct a comprehensive analysis of the package text information. The analysis is not limited to simple text analysis, but goes deep into the semantic level of the package content to identify and extract multiple sub-product names and the product entity information corresponding to each sub-product name. Among them, entity semantic information refers to the specific and identifiable product attributes or feature information contained in the text such as product names and descriptions. Each product entity information is annotated with semantic information.

[0064] Specifically, the image matching system will first use the product graph model to analyze the package text information to obtain multiple sub-product names. Figure 2B Assuming that the package information is "Single Person Package A: Full Braised Pork Roll + Chicken Nuggets + French Fries + Drink (Large), Choose One of Three Drinks: Coke, Sprite, Fanta, Made with Top-Quality Chicken, Delicious and Affordable!", the sub-product names that can be obtained by analyzing the package information using the product graph model are "Full Braised Pork Roll", "Chicken Nuggets", "French Fries", "Drink (Large)", "Drink", "Cola", "Sprite", and "Fanta", and the product graph model can also identify that "Drink (Large)" and "Drink" are synonyms, so only one of them can be analyzed in the subsequent analysis, and the hierarchical relationship between "Drink" and "Coke", "Sprite", and "Fanta" can be identified; further, it can also identify that "chicken" is the ingredient of the sub-product "chicken nuggets", so as to accurately select which content to use as the identified sub-product name.

[0065] Considering that the package products are composed of multiple independent products, and each independent product is recorded in the merchant's store, therefore, in the embodiments of the present application, in combination with the product graph model precipitated by the platform, the sub-products after disassembling the package products are mounted to the products (item and sku) in the same store, so as to realize the corresponding association between the package products and the single products in the same store, in order to better analyze the package attributes, such as: ingredients, flavors, cooking methods, standard dishes, categories, prices, etc. Therefore, for each sub-product name, the illustration system will determine the matching target sample product information for the sub-product name in multiple sample product information. The target sample product information has a high similarity in character composition with the sub-product name. The characters of the target product name that make up the target sample product information are included in the sub-product name. The target sample product information is an independent product in the merchant's store that matches the sub-product name. For example, see Figure 2C , if the sub-product name is "Sichuan-Hunan Style Yu-Shiang Shredded Pork", the target sample product information determined for this sub-product name can be "Yu-Shiang Shredded Pork". The characters of the name that make up this target sample product information are all included in the sub-product name, and "Yu-Shiang Shredded Pork" is used as a standard dish for reasoning. Since there are already corresponding entity labels for the products in these stores, the illustration system will obtain at least one first entity label corresponding to the target sample product information. Continuing to see Figure 2C , the labels "bamboo shoots", "pork", "sweet and spicy", "fried" that have been associated with "Yu-Shiang Shredded Pork" can be obtained as the first entity labels, and "Yu-Shiang Shredded Pork" itself can also be used as a first entity label; further, considering that the sub-product names identified in the package products may carry some descriptive words. For example, the identified sub-product name is "Sichuan-Hunan Style Yu-Shiang Shredded Pork", while the product in the merchant's store is "Yu-Shiang Shredded Pork", then "Sichuan-Hunan" is the descriptive word, and this descriptive word will actually also have a corresponding entity label. Therefore, in order to realize the complete analysis of the sub-product name, the illustration system will determine the characters to be analyzed in the sub-product name. The characters to be analyzed are the other characters in the sub-product name except for the characters that make up the target product name. The illustration system will obtain at least one second entity label that matches the characters to be analyzed, and use at least one first entity label and at least one second entity label as the product entity information corresponding to the sub-product name. Continuing to see Figure 2C , if the characters to be analyzed are "Sichuan-Hunan", the second entity labels obtained can be "Sichuan cuisine", "Hunan", etc. In this way, by comprehensively obtaining the first entity labels and the second entity labels, the entity labels obtained for the sub-product name "Sichuan-Hunan Style Yu-Shiang Shredded Pork" are "bamboo shoots", "pork", "sweet and spicy", "fried", "Yu-Shiang Shredded Pork", "Sichuan cuisine", "Hunan", and these entity labels constitute the product entity information corresponding to the sub-product name "Sichuan-Hunan Style Yu-Shiang Shredded Pork". Among them, Figure 2CThe analysis process of the sub-product names "Small Bowl of Shredded Pork with Fish Flavor" and "(Appetizer) Shredded Pork with Fish Flavor for One Person" shown in the figure is similar to the analysis process of the above-mentioned sub-product name "Sichuan and Hunan Shredded Pork with Fish Flavor", and will not be repeated here.

[0066] Through the above steps, the image matching system can obtain the product entity information corresponding to each sub-product name, thereby constructing a rich data set containing multiple sub-product names and multiple product entity information corresponding to each sub-product name, which greatly improves the depth and breadth of package analysis and enables the system to analyze package attributes more carefully.

[0067] 2013. Determine the product information database corresponding to the merchant's store in the server, obtain the product image matching each sub-product name in the product information database, use the multimodal visual model to perform visual analysis on the product image corresponding to each sub-product name, and obtain the product attributes corresponding to each sub-product name.

[0068] After attaching each sub-product name to the corresponding product, the image matching system will obtain product images matching each sub-product name in the merchant's store, and use a multimodal visual model to perform visual analysis on each product image to obtain the product attributes corresponding to each sub-product name.

[0069] In the process of actual application, SFT (Supervised Fine-Tuning, adapted to the package splitting task) can be performed based on the VLM multimodal visual model to build a CoT (Chain of Thought) chain for comprehensive detection and identification of product images and product attributes. This process is equivalent to a preliminary but efficient pre-labeling of product information. Among them, SFT is the process of supervised fine-tuning of a large language model, which optimizes the model by using labeled data so that it can perform specific tasks more accurately. This process generally includes adjusting the parameters of the pre-trained model to improve its performance in a specific field or task, and is usually used to improve the practicality and reliability of the model. In an embodiment of the present application, SFT is designed to adapt to the package splitting task, that is, to accurately distinguish and identify each sub-product and its attributes in the product image, which not only significantly improves the accuracy and robustness of the model in complex product recognition scenarios, but also effectively reduces the cost and time consumption of manual labeling, and improves overall work efficiency.

[0070] 2014. Based on the product attributes corresponding to each sub-product name and the product entity information, an initial product set is constructed.

[0071] After obtaining the product entity information corresponding to each sub-product name through the above process, the illustration system will build an initial product set based on the product attributes and product entity information corresponding to each sub-product name. The initial product set not only contains detailed information about each sub-product, but also clearly shows the subordinate or corresponding relationship between the sub-product and the product entity through the association mechanism, thus realizing the comprehensive integration of product information.

[0072] 2015. Output the initial product set according to a predetermined structured format to obtain a sub-product set.

[0073] A structured format refers to an output template with a clear hierarchy, field definitions, and data types, which ensures the accuracy and consistency of information and facilitates subsequent data processing and analysis.

[0074] Therefore, in the embodiment of the present application, the image matching system will perform fusion filtering based on pre-set rules, output the initial product set in a predetermined structured format, obtain a sub-product set, and output structured information of the sub-product set in a streamlined manner. The sub-product set not only retains the key attributes and associations of the products, but also removes redundant and repeated information through fusion filtering, making the data clearer, easier to understand and use.

[0075] To sum up, the process of splitting package products is summarized as follows: Figure 2D For text information such as product names and product descriptions, the product graph model is used to extract entity semantic information; for product images that can describe product attributes, a multimodal visual model is used for visual analysis, including analysis of perspective, category, and specification. Finally, the results of the two model analyses are fused and structured information is output to obtain a sub-product set.

[0076] 202. Determine multiple sub-product names included in the sub-product set, determine the product information library corresponding to the store to which the package text information belongs in the server, and extract the product picture corresponding to each sub-product name in the product information library, perform a completeness check on the product body in each extracted product picture, and when it is detected that the completeness of a product body is lower than the completeness threshold, perform an extension operation on the product picture where the product body is located.

[0077] In order to optimize the quality of illustrations in the sub-product set and solve problems such as no pictures for sub-products, inconsistency between pictures and texts, poor quality of product pictures, and inconsistent perspectives, the embodiment of the present application first clarifies the multiple sub-product names contained in the sub-product set, and extracts corresponding product illustrations for each sub-product name in the store to which the package text information belongs. These illustrations will serve as an important part of the visual presentation of the product, enhance the authenticity and credibility of the illustrations, and help improve consumers' shopping confidence and satisfaction.

[0078] Among them, in order to ensure the quality of the product illustration, the embodiment of the present application pays special attention to the problem of whether the main body of the product is incomplete, and the integrity of the main body of the product will be detected for each extracted product illustration. The integrity of the main body of the product refers to the integrity and clarity of the main body of the product in the illustration, which is directly related to whether the illustration can accurately reflect the true appearance of the product. During the detection process, the illustration system sets a completeness threshold to determine whether the main body of the product has reached sufficient integrity. When it is detected that the integrity of a certain product body is lower than the threshold, the illustration system will perform an extension operation on the product illustration where the main body of the product is located, and improve the integrity and clarity of the main body of the product in the illustration through image processing techniques such as magnification, contrast enhancement, filling in missing parts, etc. In the actual application process, large model tools (such as LlaMa) can be used for integrity repair. Large model tools such as LlaMa have powerful image generation and repair capabilities, and can intelligently fill or repair the incomplete main body of the product according to the contextual information of the illustration, so as to obtain a more complete and clear product illustration.

[0079] In this way, through a series of measures such as giving priority to recommending similar sub-product pictures, checking the integrity of the main body of the product, and using large model tools for integrity repair, we have effectively solved the problems of no pictures for sub-products, inconsistent pictures and texts, poor quality of product pictures, and inconsistent perspectives. This comprehensive approach not only improves the quality and accuracy of product pictures, but also enhances the matching and authenticity of pictures and sub-product names, ensuring that the subsequently generated package pictures can provide consumers with more intuitive and accurate product information display.

[0080] 203. Perform a cutout operation on each product picture to obtain a cutout result of each product picture, and score the quality of the cutout result of each product picture. According to the scoring result, select a preset number of sub-product pictures for each sub-product name to obtain multiple sub-product pictures corresponding to the multiple sub-product names.

[0081] Considering the product distribution Figure 1Generally, it is with an atmosphere background, and the elements are not pure enough, and the matching set needs to be based on a transparent image to improve the quality of the image. Therefore, in the embodiment of the present application, the picture matching system will first perform a cutout operation on each product picture to obtain the cutout result of each product picture. Cutout is to use image processing technology to separate the product from the background, retain only the product itself, and remove unnecessary background elements to highlight the main body of the product, thereby improving the purity and visual appeal of the product display. Next, the picture matching system uses an image quality evaluation algorithm to score the quality of each cutout result. The scoring process will comprehensively consider the accuracy of the cutout, the smoothness of the edge processing, the authenticity of the color restoration, and the harmony of the overall visual effect. Multiple dimensions to ensure the objectivity and accuracy of the score. According to the scoring results obtained, the picture matching system will select a preset number of sub-product pictures for each sub-product name, and obtain multiple sub-product pictures corresponding to multiple sub-product names. Among them, the picture matching system will set reasonable screening criteria, such as selecting the pictures with the top scores, to ensure that each sub-product name corresponds to a series of high-quality and diverse sub-product pictures. The following describes the process of selecting a preset number of sub-product images for each sub-product name, which specifically includes steps 2031 to 2033:

[0082] 2031. For each sub-product name, compare the scoring result corresponding to the sub-product name with the result threshold.

[0083] For each sub-product name, the image matching system will compare the scoring result corresponding to the sub-product name with the result threshold. The result threshold is a pre-set scoring standard used to measure whether the cutout quality is qualified or not.

[0084] In the embodiment of the present application, the illustration system sets a relatively high scoring threshold, such as 8.0 points, as a benchmark for judging whether the cutout quality is high. This is based on strict consideration of multiple key indicators such as cutout accuracy, edge processing, and color restoration. It aims to ensure that only those cutout results that truly meet high-quality standards can be recognized as "approved and usable". It not only ensures the professionalism and attractiveness of the product illustrations, but also eliminates those illustrations that do not meet the quality standards and may affect consumers' shopping experience through a strict screening mechanism.

[0085] 2032. When the scoring result is greater than or equal to the result threshold, a perspective detection is performed on the product image corresponding to the sub-product name, and a preset number of sub-product images are selected for the sub-product name based on the obtained perspective detection result.

[0086] For product pictures corresponding to sub-product names whose scoring results are greater than or equal to the preset result threshold, the picture system will use perspective detection technology to identify their perspective type and obtain perspective detection results. This perspective detection result not only helps to understand the display angle of the product pictures, but also provides a key basis for the subsequent selection of sub-product pictures.

[0087] In order to evaluate the viewing angle detection results, a viewing angle priority is set in the embodiment of the present application. According to the user's viewing angle logic, the viewing angle priority is set as follows: the oblique view is considered preferred because it is more in line with the user's viewing habits; if the oblique view is missing, the top view is searched instead; if neither is available, the front view is used as an alternative. The viewing angle priority setting is based on the designer's rich experience and deep understanding of user preferences, and is intended to ensure that the selected sub-product illustrations can maximize the user's attention and enhance the product display effect.

[0088] Therefore, after obtaining the perspective detection result, the picture matching system will identify whether the perspective detection result indicates that the product picture is a front view. Among them, when the perspective detection result indicates that the product picture is a front view, in order to ensure the picture matching effect, the picture matching system will search in the preset product picture library whether there is a specified preset product picture that matches the sub-product name and product picture and has an oblique perspective. The preset product picture library contains transparent background resources. The transparent background only shows the product itself and completely removes the background elements. With its simple and pure presentation method, the subsequent picture matching system can easily combine multiple products together without worrying about background interference or color conflict, thereby creating a package picture that is both beautiful and attractive, with good convenience and flexibility. If the search determines that there is a specified preset product picture, the picture matching system will use the specified preset product picture as a sub-product picture; at the same time, the picture matching system will continue to search for the first other preset product picture that matches the sub-product name and product picture at each perspective in the preset product picture library according to the perspective priority, and use the searched first other preset product picture and the specified preset product picture as the preset number of sub-product pictures to ensure that the preset number of sub-product pictures is obtained. On the contrary, if the search determines that the specified preset product image does not exist, considering that the product image is actually the most realistic image provided by the merchant, the image system will retain the product image and use it as a sub-product image. At the same time, the system will continue to search the preset product gallery for a second other preset product image that matches the sub-product name and product image at each perspective according to the perspective priority, and use the searched second other preset product image and product image as a preset number of sub-product images, thereby ensuring that the preset number of sub-product images is supplemented according to the perspective priority search.

[0089] When the perspective detection result indicates that the product picture is not a front view, the product picture is most likely an oblique view. Therefore, the picture system will take the product picture as a sub-product picture, and search the preset product gallery for a third other preset product picture that matches the sub-product name and product picture at each perspective according to the perspective priority. The searched third other preset product pictures and product pictures will be used as a preset number of sub-product pictures to construct a complete set of preset number of sub-product pictures.

[0090] In this way, through this integrated perspective detection and optimization strategy, the embodiment of the present application can not only ensure the perspective diversity and attractiveness of the product pictures, but also intelligently select sub-product pictures with better perspectives according to user preferences and product characteristics, thereby improving the visual effects of the package pictures.

[0091] 2033. When the scoring result is less than the result threshold, determine the perspective priority, and in the preset product gallery, select preset product images that match the sub-product names and are equal to the preset number as the preset number of sub-product images according to the perspective priority.

[0092] When the scoring result is less than the result threshold, the current cutout recognition can be determined. On the premise that the dishes are similar, the image matching system will select preset product images that match the sub-product names and are equal to the preset number in the preset product gallery according to the perspective priority as the preset number of sub-product matching images. It not only considers the matching degree of the sub-product names, but also takes into account the priority of the shooting perspective, to ensure that the selected preset product images are in line with the product characteristics and can be presented at a better perspective.

[0093] In order to further solve the problem of large differences between the original product image and the searched product image due to failed cutouts, the embodiment of the present application introduces a picture and text search mechanism. The image matching system will perform a picture and text search in the preset product gallery based on the cutout product image (even if incomplete) combined with the sub-product name, thereby avoiding significant deviations between the searched product image and the original product. By integrating image and text information, the accuracy and relevance of the search results are significantly improved.

[0094] During the image and text search process, when multiple candidate preset product images that match the sub-product name and product illustration are searched for a certain perspective in the preset product gallery, that is, there are multiple candidate preset product images under the same perspective, the illustration system will use a similarity sorting algorithm to calculate the image similarity between each candidate preset product image and the product illustration, and use the candidate preset product image with the greatest image similarity among the multiple candidate preset product images as the other preset product images that match the sub-product name and product illustration searched under the perspective. Among them, the similarity sorting algorithm can accurately quantify the similarity between images by comparing image features, color distribution, texture information, etc., ensure the high visual consistency between the selected transparent base image and the original product image, ensure that a high-quality transparent image corresponding to each sub-product in the package is obtained, and continue to generate the complete package image in the last step.

[0095] 204. Determine at least one style layout template that matches the package text information.

[0096] In order to improve the quality of illustrations, enrich design styles and speed up delivery, the embodiment of the present application solidifies a batch of style layout template designs for pre-recommendation based on the visual experience of high-quality package pictures with high daily sales volume. The style layout template design covers a variety of scenarios under different numbers of package sub-items, aiming to provide users with diverse choices. However, in order to avoid the generated package pictures being completely consistent with the known package pictures, in the embodiment of the present application, the picture matching system will first identify the design element information of the sample package pictures, form text description information for describing its design style, and use a large model to rewrite these text description information to form a preset style layout template, so that the package pictures generated subsequently using these preset style layout templates not only inherit the high-quality style of the sample package pictures, but also have some unique style characteristics compared to the sample package pictures. The process of generating a preset style layout template is described below, including the following steps 2041 to 2042:

[0097] 2041. Obtain multiple sample package illustrations and determine the design element information of each sample package illustration.

[0098] First, a batch of sample package illustrations are selected. In the embodiment of the present application, the illustration system will deeply analyze the design element information of each sample package illustration, including key elements such as color matching, layout structure, and font style.

[0099] 2042. Use a multimodal visual model to process each sample package picture and corresponding design element information to obtain text description information corresponding to each sample package picture.

[0100] Subsequently, the embodiment of the present application uses a multimodal visual model to perform in-depth processing on these sample package pictures and their design element information, and extracts text description information that can accurately describe the sample design style. These text description information is used to describe the design style of the corresponding sample package pictures, which not only records the visual features of the sample in detail, but also is associated with the entity tags that match the corresponding sample package pictures, and establishes a close connection with the corresponding sample package pictures through the entity tags.

[0101] 2043. Input the text description information corresponding to each sample package picture into the large language model for rewriting, and obtain the rewritten multiple text description information output by the large language model.

[0102] Next, the embodiment of the present application inputs the text description information corresponding to each sample package picture into the large language model for a rewrite, and creatively rewrites these text description information with the help of the large language model to obtain the rewritten multiple text description information output by the large language model. The large language model is also LLM (Large Language Model), which refers to a large-scale pre-trained language model with a large number of parameters. Such models are usually based on deep learning architectures, such as Transformer, and they learn language structures and patterns by performing unsupervised pre-training on massive text data. LLM can perform various natural language processing tasks, such as text generation, question and answer, translation, etc., without the need for additional specific task training. In the embodiment of the present application, LLM, with its powerful language generation ability, can generate multiple rewritten versions with different styles and novel and unique on the basis of maintaining the original meaning. Among them, when rewriting, the picture matching system will set some prompt words related to rewriting, and the prompt word is a guiding text used when interacting with the large language model, and can guide the model to generate relevant responses or complete specific tasks through specific prompt words. The prompt can be a simple question, a piece of explanatory text, or a more complex instruction. The purpose is to enable the model to produce meaningful output based on these prompts. The following example illustrates the process of rewriting text description information:

[0103] Assume that the original text description information is "This style layout template adopts a fresh and natural style, with bright green and warm yellow as the main colors, and simple and crisp lines, creating a relaxing and pleasant dining atmosphere";

[0104] If you want to strengthen the emotional atmosphere so that the meal set pictures generated by the template are more cozy and comfortable than the sample meal set pictures, the prompt words you set may be "Please strengthen the emotional atmosphere in the description of the style layout template so that customers can feel the warmth and comfort of home." In this way, the rewritten text description information may be "This style layout template is centered on a cozy and comfortable family atmosphere, using soft green and warm yellow, with delicate and soft lines, to create a dining experience that makes customers feel like they are at home."

[0105] If you want to enhance the visual impact so that the set meal pictures generated by the template provide a sharper contrast than the sample set meal pictures, the prompt words you set may be "Please rewrite the description to enhance the visual impact of the style layout template and make the dishes more attractive." In this way, the text description information obtained after rewriting may be "This style layout template uses contrasting green and yellow colors, with dynamic lines and light and shadow effects, to create a mouth-watering visual feast, making the dishes look more delicious and stimulating customers' appetite."

[0106] If you want to incorporate cultural elements so that the meal set pictures generated by the template can show more cultural heritage than the sample meal set pictures, the prompt words you set can be "Please incorporate traditional cultural elements into the style layout template description to show the cultural heritage of the dishes." In this way, the rewritten text description information may be "This style layout template incorporates elements of traditional ink painting, with elegant green and warm yellow as the base tone, combined with smooth and natural lines and delicate brushstrokes, to create an antique dining atmosphere, showing the profound cultural heritage and unique oriental charm of the dishes."

[0107] If you currently want to locate the target audience and make the meal set pictures generated by the template in the future better meet the target audience's aesthetic standards than the sample meal set pictures, the prompt word you set may be "Please rewrite the description for the young consumer group to make it more in line with their aesthetics and preferences." In this way, the text description information obtained after the rewrite may be "This style layout template is designed for the young consumer group, using a fashionable and avant-garde combination of green and yellow, with simple and smooth lines and dynamic composition, to create a fashionable, vibrant and individual dining atmosphere, allowing young customers to enjoy delicious food while also experiencing a unique visual experience."

[0108] In this way, through the above-mentioned rewriting, the core elements of the original style are retained, and new expressions and visual experiences are added.

[0109] 2044. Generate a test image based on each rewritten text description information using an image generation model, rewrite the corresponding rewritten text description information according to the content displayed in the test image, and use the multiple text description information after the second rewrite as multiple first candidate text description information.

[0110] In order to further improve the accuracy and practicality of the rewritten text description information, the embodiment of the present application uses an image generation model to generate a test picture based on the rewritten text description information, and rewrites the corresponding text based on the display effect of the test picture, ensuring that the rewritten text description information can more accurately guide the image generation process and improve the quality of picture generation.

[0111] Assuming that the text description information after a rewrite is "This style layout template uses contrasting green and yellow colors, with dynamic lines and light and shadow effects, to create a mouth-watering visual feast, making the dishes look more delicious and stimulating customers' appetite", the image generation model is used to generate a test picture based on the text description information after a rewrite. The test picture should show the contrast between green and yellow, as well as dynamic lines and light and shadow effects.

[0112] Next, the generated test images are evaluated in terms of color matching, lines, light and shadow, and overall atmosphere. For example, whether the contrast between green and yellow is vivid enough and meets the expected visual effect; whether the lines are dynamic and whether the light and shadow effects are natural and can highlight the dishes; whether the images successfully create a mouth-watering visual feast and whether the dishes look more delicious, etc.

[0113] Then, a second rewrite is performed based on the evaluation. The colors can be refined, such as refining "contrasting green and yellow" into "contrasting dark green and bright yellow tones" to make the color description more specific and vivid; the lines and light and shadow can be refined, such as adding descriptions of "finely outlined dynamic curves" and "delicate light and shadow gradients" to make the presentation of lines and light and shadow more specific and delicate, etc., so that the text description information obtained is "This style layout template cleverly combines contrasting dark green and bright yellow tones, and creates a vivid and attractive visual experience through finely outlined dynamic curves and delicate light and shadow gradients. The dishes are more attractive against the green background, and the yellow embellishment adds a bit of liveliness and warmth. The light and shadow effects are natural and smooth, which not only highlights the three-dimensionality and layering of the dishes, but also cleverly guides the visual focus, making the customers' eyes involuntarily focus on the dishes, as if they can smell the fragrant aroma, further stimulating the customers' appetite and desire to buy."

[0114] In this way, through secondary rewriting, it can be ensured that the rewritten text description information can more accurately guide the image generation process, improve the quality of picture generation, and make it more in line with the expected visual effects and publicity needs.

[0115] 2045. Use the product graph model to construct multiple second candidate text description information, cross-recombine the multiple first candidate text description information and the multiple second candidate text description information to obtain multiple target text description information.

[0116] In addition, considering that a product graph model has been constructed before, the product graph model can accurately and in detail describe the style characteristics of each product. Therefore, the embodiment of the present application also introduces the product graph model to construct multiple second candidate text description information. The product graph model can generate a text description that is closely related to the product and has a unique style by deeply mining information such as product attributes and relationships.

[0117] For example, for a hamburger meal, candidate text description information such as "a fresh and juicy hamburger paired with crispy and delicious French fries forms a delicious combination" can be generated based on the product graph model, which not only reflects the characteristics of the product, but also increases the attractiveness of the accompanying picture.

[0118] 2046. Construct multiple preset style layout templates based on multiple target text description information, and use the entity label associated with each target text description information to annotate the generated preset style layout templates.

[0119] Finally, the image matching system cross-recombines multiple first candidate text description information and multiple second candidate text description information to obtain multiple target text description information. These target text description information not only integrates elements of multiple styles, but also establishes a corresponding relationship with the preset style layout template through entity tags. Based on these target text description information, the embodiment of the present application successfully constructs multiple preset style layout templates with unique styles, and accurately marks the templates through entity tags to facilitate subsequent use and storage.

[0120] Through the above steps 2041 to 2046, multiple preset style layout templates are created. The specific process of generating preset style layout templates is summarized as follows: Figure 2E , obtain sample package pictures, use the VLM visual model to understand the scene and content of the sample package pictures, and determine the design element information. The design element information specifically involves the style, details, element layout, color matching, clarity, light conditions, etc. of the sample package pictures. On the one hand, the design element information is sorted to obtain text description information, and the LLM model is used to rewrite the text description information once, and the SDXL model is used to generate test pictures based on each rewritten text description information. The corresponding rewritten text description information is rewritten again according to the content displayed by the test picture, and the test pictures are continued to be generated, and multiple rounds of debugging are performed. On the other hand, the commodity map model is used to construct multiple second candidate text description information, and multiple first candidate text description information and multiple second candidate text description information are cross-recombined to obtain multiple target text description information. After generating multiple target text description information, the SDXL model can also be used to generate test pictures and perform multiple rounds of debugging, and finally a style layout template is generated, and the style layout template is sent to manual review, thereby forming at least one style layout template.

[0121] In the actual application process, each preset style layout template determines the initial topological information of the layout of the main dishes, side dishes and drinks by labeling, so that based on the preset style layout template, the template that best matches the text information of the meal can be efficiently identified and matched, ensuring the diversity and applicability of the preset style layout template. In this way, when a meal is subsequently input, the template with a high matching score can be automatically recalled, and then combined with the adaptive layout to obtain the meal picture. In this way, for the text information of the meal, the picture matching system will use the recall algorithm to automatically search and filter out the preset style layout template with the highest matching degree with the content of the meal text information, as at least one style layout template in the candidate template set, which not only greatly improves the degree of automation of the picture combination, but also ensures that the final generated picture meets the aesthetic standards and can accurately convey the product information.

[0122] 205. Referring to at least one style layout template, multiple sub-product pictures are combined and laid out to obtain and output at least one set meal picture.

[0123] In the embodiment of the present application, after obtaining at least one style layout template that matches the text information of the package, multiple sub-product pictures can be combined and laid out with reference to the at least one style layout template to obtain and output at least one package picture. Specifically, considering that each sub-product will correspond to multiple sub-product pictures, the picture system will determine the multiple sub-product names included in the sub-product set, randomly select a sub-product picture for each sub-product name from the multiple sub-product pictures, and use each style layout template to combine and lay out the sub-product pictures currently selected for each sub-product name to obtain a package picture generated based on the currently selected sub-product picture, and randomly select a new sub-product picture for each sub-product name and use each style layout template to generate a package picture, until each sub-product picture in the multiple sub-product pictures is traversed to obtain at least one package picture and output, so that each sub-product picture corresponding to each sub-product will participate in the picture generation process, and multiple sets of pictures are generated for merchants to choose from, meeting the various needs of merchants.

[0124] However, in the process of actual application, the applicant realized that it is difficult to obtain high-quality package pictures based on the style layout template of the preset pit layout, because the sub-product pictures themselves are strange and varied in shape and size, and are prone to disproportion and mutual covering, resulting in low delivery rate. Therefore, in the embodiment of the present application, when using each style layout template to combine and layout the sub-product pictures currently selected for each sub-product name, the picture matching system will implement intelligent layout based on the traditional greedy algorithm, and on the basis of the preset style layout template, solve the limitations and make the generated package pictures more harmonious. The following describes the process of combining and laying out the sub-product pictures currently selected for each sub-product name using each style layout template, including the following steps 2051 to 2055:

[0125] 2051. For each style layout template, identify preset layer information of the style layout template.

[0126] In the embodiment of the present application, for each style layout template, the drawing system will perform a detailed analysis to identify the preset layer information in the style layout template, which includes but is not limited to the initial layout center of the layer, the main area of ​​the main object, and the relative distance between the layers and other key parameters. These parameters together constitute the basic framework of the template, providing precise guidance for subsequent layer initialization and layout adjustment.

[0127] 2052. According to the preset layer information and the sub-product pictures currently selected for each sub-product name, perform layer information initialization operation and layout initialization operation to obtain a layout framework.

[0128] The image matching system will perform a series of detailed layer information initialization operations and layout initialization operations based on the sub-product images selected for each sub-product name in the current product package and combined with the preset layer information.

[0129] During this process, the graphics system will first calculate the layer parameters based on the preset layer information, scale the canvas to the preset bounding box, and use the layer parameters and the scaled canvas to build the initial frame to complete the layer information initialization operation to ensure that all elements can be adapted.

[0130] Subsequently, the image matching system will make necessary adjustments to the initial frame with reference to the size and shape of the sub-product images selected for each sub-product name, including canvas expansion, layout center adjustment, and determination of the order of pasting. That is, the canvas expansion operation is performed on the initial frame with reference to the sub-product images currently selected for each sub-product name, and the initial layout center in the initial frame is adjusted according to the expanded canvas to obtain the layout center, and the order of pasting of the sub-product images currently selected for each sub-product name is determined with reference to the position and distance between the layers indicated by the initial frame, and the layout initialization operation is completed, thereby obtaining a layout framework that conforms to aesthetic principles and can fully display the characteristics of the product.

[0131] 2053. Determine the mapping order indicated by the layout framework, select the first sub-product illustration to be laid out from the sub-product illustrations currently selected for each sub-product name according to the mapping order, and update the layer position information of the layer where the first sub-product illustration is located within the layout framework according to the canvas scaling size indicated by the layout framework.

[0132] After getting the layout framework, the graphics system will determine the order of the graphics indicated by the layout framework, and embed the sub-product graphics into the corresponding layers one by one according to the order of the graphics. In order to ensure that each sub-product graphics can be presented in the best way, the graphics system will determine the layout position of each layer.

[0133] Among them, first, the picture matching system will select the first sub-item picture to be laid out from the sub-item pictures currently selected for each sub-item name according to the order of pictures, and update the layer position information of the layer where the first sub-item picture is located in the layout frame according to the canvas zoom size indicated by the layout frame. The first sub-item picture can be the visual focus or core element of the entire set meal picture, such as the picture of the main dish. The picture matching system will accurately update the position information of the layer where the first sub-item picture is located according to the canvas zoom size indicated in detail in the layout frame.

[0134] The canvas scaling size is designed to adapt to different display scenarios and screen sizes. Updating the position of the first sub-product image according to the canvas scaling size can ensure that the position and size of the first sub-product image in the layout frame are just right, neither too crowded nor too empty. By precisely controlling the position and size of the first sub-product image in the layout frame, the visual focus of the entire package image can be ensured, while maintaining the harmony and balance of the overall layout.

[0135] 2054. Continue to determine the second sub-product illustration to be laid out in the sub-product illustrations currently selected for each sub-product name in the order of pasting, determine the layer center of the layer where the second sub-product illustration is located, determine the position point to be laid out on the straight line between the layout center indicated by the layout frame and the layer center, update the layer position information of the layer where the second sub-product illustration is located in the layout frame with reference to the position point to be laid out, and continue to determine the third sub-product illustration to be laid out indicated by the layout frame in the sub-product illustrations currently selected for each sub-product name, determine the position point to be laid out for the third sub-product illustration and update the layer position information, until the layer position information update process of the sub-product illustration currently selected for each sub-product name is completed, and the layout frame to be used is obtained.

[0136] After the position of the first sub-product illustration is determined, the illustration system will continue to select the second sub-product illustration and determine the position for the second sub-product illustration. During this process, the illustration system will comprehensively consider multiple factors such as the relative position between the layer center and the layout center, the overlap between layers, and the overlap ratio, and iterate and compare repeatedly until the most suitable layout position is found. In particular, when there is an overlap between two layers, the system will adjust the layout position according to the overlap ratio to avoid visual conflicts and confusion, ensuring that the final illustration is both beautiful and practical. The following describes the process of determining the layout position point for the second sub-product illustration:

[0137] First, the illustration system will set the layout center as the initial left endpoint of the layer where the second sub-product illustration is located. This setting is based on the overall planning of the layout framework and ensures that the starting point of the layout process is clear.

[0138] Next, the illustration system will calculate the length of the line segment between the layout center and the layer center based on a preset multiple, which is pre-set according to the layout aesthetics and product display requirements. Then, on the straight line between the layout center and the layer center, the illustration system will find a point whose distance from the left endpoint is exactly equal to the product of the preset multiple and the line segment length. This point is set as the right endpoint of the layer. Through this step, the illustration system has drawn a preliminary left and right boundary for the second sub-product illustration on the layer.

[0139] Subsequently, the image matching system will calculate the middle point based on the distance between the left and right endpoints. This point serves as a potential layout center and provides a basis for subsequent layout adjustments. Next, the system will query a specified layer in the layout framework with this middle point as the layer center, and calculate the overlapping pixel area between the specified layer and the layer where the second sub-product image is located. This step is to detect whether there is a visual conflict or overlap between the two layers to ensure that the final layout effect is harmonious and unified.

[0140] If the overlapping pixel area is not 0, it means that there is an overlap between the two layers, and the illustration system will use the overlapping pixel area to calculate the overlap ratio between the two layers. This ratio reflects the degree of visual overlap between the two layers and is an important basis for adjusting the layout position. Therefore, the illustration system refers to the overlap ratio to determine the position point to be laid out for the second sub-product illustration. Among them, a ratio threshold is set in the illustration system. The ratio threshold is pre-set according to the layout aesthetics and product display requirements. The illustration system will obtain the ratio threshold and compare the overlap ratio with the ratio threshold. When the overlap ratio is less than the ratio threshold and greater than 0, it means that the overlap between the two layers is within an acceptable range. At this time, the illustration system will form the position point to be laid out based on the left endpoint and the right endpoint. When the overlapping ratio is greater than the ratio threshold, it means that the degree of overlap between the two layers is too high. The illustration system will adjust the positions of the left and right endpoints again, and use the middle point as the left endpoint of the layer where the second sub-product illustration is located. The updated left endpoint is used to redetermine the right endpoint and recalculate the new middle point. Based on the new middle point, the overlapping ratio of the layer where the second sub-product illustration is located is recalculated and compared until the calculated overlapping ratio is less than the ratio threshold and greater than 0. The position point to be laid out is formed based on the current left endpoint and the current right endpoint to ensure that a position point is found that meets the layout requirements and maintains visual balance.

[0141] If the overlapping pixel area is 0, it means that there is no overlap between the two layers. The layers may be split and not coordinated enough. A certain overlap is required to ensure the beauty of the overall composition. Therefore, the image matching system will use the middle position point as the right endpoint of the layer where the second sub-product image is located, and use the updated right endpoint to redetermine the left endpoint and recalculate the new middle position point, and recalculate the overlapping pixel area for the layer where the second sub-product image is located based on the new middle position point until the calculated overlapping pixel area is not 0. The overlap ratio between the two layers is calculated using the currently calculated overlapping pixel area, and the position point to be laid out is determined for the second sub-product image with reference to the overlap ratio, ensuring that a position point that meets the layout requirements and avoids visual conflicts is found. Among them, the process of determining the position point to be laid out according to the overlap ratio is consistent with the process described above, and will not be repeated here. In summary, the process of determining the position point to be laid out is summarized as follows: See. Figure 2F, according to the layer information, the initial left and right endpoints of the layer where the sub-product illustration is located, and the middle position point is determined according to the distance between the left and right endpoints; then the overlapping pixel area between the specified layer and the layer where the sub-product illustration is located is calculated. If the overlapping pixel area is not 0, the overlapping ratio between the two layers is calculated. When the overlapping ratio is less than the ratio threshold and greater than 0, the position point to be laid out is formed based on the left and right endpoints. When the overlapping ratio is greater than the ratio threshold, the middle position point is used as the left endpoint of the layer where the sub-product illustration is located, and the updated left endpoint is used to redetermine the middle position point and calculate until the overlapping ratio is less than the ratio threshold and greater than 0, and the position point to be laid out is formed based on the current left endpoint and the current right endpoint. If the overlapping pixel area is 0, the middle position point is used as the right endpoint of the layer where the sub-product illustration is located, and the updated right endpoint is used to redetermine the middle position point and calculate until there is an overlapping pixel area between the two layers, and the overlapping ratio is less than the ratio threshold and greater than 0, and the position point to be laid out is formed based on the current left endpoint and the current right endpoint.

[0142] In this way, through the above process, a position point to be laid out can be accurately determined for the second sub-product picture according to the layout aesthetics and product display requirements, which can avoid visual conflicts and maintain the overall layout harmony. Moreover, through repeated iterations and adjustments, the picture system can determine a better position point to be laid out for each sub-product picture, ensuring that the final picture is both beautiful and practical, providing strong visual support for the product package. 2055. Reorder the layers in the layout framework to be used according to the layer index, and refer to the reordered layout framework to be used to map the sub-product pictures currently selected for each sub-product name, so as to obtain the package pictures generated based on the currently selected sub-product pictures.

[0143] In the embodiment of the present application, a reordering mechanism is set in the picture matching system, which can automatically adjust the layers according to the layer index to ensure that the final picture matching not only conforms to the aesthetic principles but also clearly conveys the information of the product package. After the reordering process is completed, the picture matching system will refer to the updated layout framework to be used and perform mapping on the sub-product matching pictures currently selected for each sub-product name, thereby generating a picture matching the package that is both beautiful and practical.

[0144] Thus, in the process from step 2051 to step 2055, the layout template is quickly matched and adaptively adjusted by presetting layer information and automatic initialization operations, which greatly improves the efficiency of picture generation; secondly, by finely determining the order of pasting and the position points to be laid out, it is ensured that each sub-product picture can be presented in a better way, thereby improving the overall quality of the picture; finally, through intelligent reordering mechanism and pasting processing, a package picture is generated that conforms to aesthetic principles and can clearly convey product information, providing strong support for product promotion and sales. In summary, the process of combining multiple sub-product pictures to generate a package picture is summarized as follows: See Figure 2G , identify the preset layer information of the style layout template, perform layer information initialization operations and layout initialization operations, and obtain the layout framework. For each sub-product picture, determine whether it is the first sub-product picture. If it is the first sub-product picture, update the layer position information of the layer where the first sub-product picture is located in the layout framework according to the canvas scaling size indicated by the layout framework; then continue to determine the position point to be laid out on the straight line between the layout center and the layer center indicated by the layout framework for sub-product pictures that are not the first, and update the layer position information of the layer where the current sub-product picture is located in the layout framework with reference to the position point to be laid out, until each sub-product picture is traversed. Finally, execute the redrawing process, reorder the layers in the layout framework to be used according to the layer index, and refer to the layout framework to be used after the reordering process to map the sub-product pictures currently selected for each sub-product name to obtain the package pictures of the new layout.

[0145] Among them, see Figure 2H If you generate package pictures directly according to the style layout template, you may generate Figure 2H In the set meal picture shown in Figure ①, there is unreasonable overlap between the sub-item pictures, and the composition is not coordinated enough; however, after using the process described above to make intelligent layout adjustments, a Figure 2H In the package pictures shown in Figure ②, there is overlap between the sub-product pictures, but the overlap is reasonable and the composition is beautiful. It should be noted that at least one generated package picture can be sent to the platform's reviewers first, and the reviewers will manually determine whether the picture effect is feasible. When the reviewers determine that the effect is feasible, it will be sent to the merchant for reference. Among them, if the reviewer has adjustment instructions, the picture system will adjust the generated package picture according to the reviewer's requirements, and send it to the merchant for reference after the reviewer confirms it.

[0146] In summary, the logical process of the technical solution of this application is summarized as follows:

[0147] See also Fig.2I, get the package input by the merchant, parse the sub-products included in the package, and get the style layout template according to the number of sub-products and the package category. Then, check whether there are corresponding pictures for each sub-product in the merchant's store. If not, search for transparent background pictures in the preset gallery. When searching, prioritize the oblique view. If there is no oblique view, search for the top view. If there is no top view, search for the front view. Use the searched picture as the sub-product picture of the corresponding sub-product. If there is a picture corresponding to the sub-product in the store, cut out the picture of the sub-product. Before cutting out the picture, check the completeness of the main body of the product. If it is incomplete, complete it. After the cutout, the quality of the cutout result needs to be scored. If the cutout quality is low, the transparent base image is searched in the preset gallery, and the above principles are also followed during the search; if the cutout quality meets the requirements, the perspective detection of the sub-product's matching image is continued to detect whether it is a front view. If it is not a front view, it is considered that the matching image is an oblique view and can be used as the matching image of the sub-product; if it is a front view, it cannot be used, and it is necessary to search for an oblique transparent base image in the preset gallery. Similarly, if an oblique view can be searched during the search, the searched oblique view is used. If no oblique view is found, the sub-product view that is the front view is used. Among them, 3 sub-product matching images need to be selected for each sub-product according to the above process. Next, the matching system uses the acquired style layout template to combine the acquired sub-product matching images, optimizes the layout during the combination process, generates a package matching image, and sends the package matching image to the reviewer for review, and the complete package matching image is generated.

[0148] In general, the technical solution of the present application is based on the combination of VLM, LLM large models and local dish knowledge graph NER model, which can efficiently and accurately extract high-quality sub-product splitting structured information from the package name and product description information, and make the text entity decomposition more accurate and the raw image result better through the consistency of the association between themes; further, based on the UED preset experience template, controllability is guaranteed to a certain extent, and at the same time, combined with the intelligent layout technology, the preset layout is dynamically adjusted to optimize the visual presentation of the content; further, based on the SDXL large model, the ControNet technology is integrated to integrate multiple sub-product elements into the artistic background generation, which has certain novelty in function compared with the ordinary multi-commodity rigid stacking raw image method; moreover, flexible content expression is realized, and an efficient production review mechanism is added (the combination of VLM machine review + human review is introduced). Through the construction of a fully automated processing chain, production efficiency is greatly improved, manpower investment is reduced, development cycle and budget cost are greatly shortened, and the controllability and consistency of the original image in the existing technology are solved, which greatly simplifies the delivery process of package products in merchant operations, and has broad application prospects.

[0149] The method provided in the embodiment of the present application determines the package text information for which the illustration is to be generated, parses the package text information, obtains a sub-product set corresponding to the package text information, selects multiple sub-product illustrations based on the sub-product set, and determines at least one style layout template matching the package text information, combines and lays out the multiple sub-product illustrations with reference to the at least one style layout template, obtains and outputs at least one package illustration, and can automatically parse and determine the names of the sub-products included in the package text information and select an independent illustration for each sub-product name, and combines the independent illustrations using a style layout template matching the package text information to generate a complete package illustration. Merchants do not need to prepare illustrations for the packages by themselves, and while simplifying the illustration process for the packages and improving the illustration efficiency, the style layout template is used to ensure the consistency of the illustration style and the controllability of the quality, thereby improving the overall visual effect of the package illustration.

[0150] Further, as Figure 1 In a specific implementation of the method, the present application embodiment provides a set meal image generation device, such as Figure 3 As shown, the device includes: an analysis module 301, a determination module 302 and a combination module 303.

[0151] The parsing module 301 is used to determine the text information of the meal package for which the picture is to be generated, parse the text information of the meal package, and obtain the sub-product set corresponding to the text information of the meal package;

[0152] A determination module 302 is used to select a plurality of sub-product pictures based on the sub-product set, and determine at least one style layout template that matches the package text information;

[0153] The combination module 303 is used to combine and layout the multiple sub-product pictures with reference to the at least one style layout template, obtain at least one set meal picture and output it.

[0154] In a specific application scenario, the parsing module 301 is used to respond to a merchant's request for generating an illustration, determine the name of the product package indicated by the illustration generation request, and use the text information associated with the product package name as the package text information for the illustration to be generated; use the product graph model to extract entity semantic information from the package text information to obtain multiple sub-product names and product entity information corresponding to each of the sub-product names, wherein each of the product entity information is annotated with semantic information; determine the product information library corresponding to the merchant's store in the server, and obtain product images matching each of the sub-product names in the product information library; use a multimodal visual model to perform visual analysis on the product images corresponding to each of the sub-product names to obtain product attributes corresponding to each of the sub-product names; construct an initial product set based on the product attributes corresponding to each of the sub-product names and the product entity information; output the initial product set in a predetermined structured format to obtain the sub-product set.

[0155] In a specific application scenario, the parsing module 301 is also used to match multiple sample product information, analyze the product attributes corresponding to each of the sample product information, and construct the product graph model based on the multiple sample product information and the product attributes corresponding to each of the sample product information; set multiple sample questions, construct an answer template corresponding to each of the sample questions, and use the multiple sample questions and the answer template corresponding to each of the sample questions to guide the training of the initial visual model to obtain the multimodal visual model, wherein the multiple sample questions include text questions and image questions.

[0156] In a specific application scenario, the parsing module 301 is used to match the products connected to the platform where the merchant's store is located, and use the product information of the matched multiple products as the multiple sample product information; use a pre-trained deep language model to adaptively fine-tune each of the sample product information; use a two-layer bidirectional long short-term memory network to encode each of the sample product information that has been adaptively fine-tuned to obtain text feature information corresponding to each of the sample product information; input the text feature information of each of the sample product information into a conditional random field for parsing to obtain at least one entity tag corresponding to each of the sample product information, and use the obtained at least one entity tag as the product attribute of the corresponding sample product information.

[0157] In a specific application scenario, the parsing module 301 is used to analyze the package text information using the product graph model to obtain the multiple sub-product names; perform the following processing on each of the sub-product names: determine the matching target sample product information for the sub-product name from the multiple sample product information, obtain at least one first entity tag corresponding to the target sample product information, and determine the characters to be analyzed in the sub-product name, obtain at least one second entity tag matching the characters to be analyzed, and use the at least one first entity tag and the at least one second entity tag as the product entity information corresponding to the sub-product name, wherein the characters constituting the target product name of the target sample product information are included in the sub-product name, and the characters to be analyzed are other characters in the sub-product name except the characters constituting the target product name; obtain the product entity information corresponding to each of the sub-product names.

[0158] In a specific application scenario, the device also includes:

[0159] A generation module is used to obtain multiple sample package pictures, determine the design element information of each sample package picture; use a multimodal visual model to process each sample package picture and the corresponding design element information to obtain text description information corresponding to each sample package picture, wherein the text description information is used to describe the design style of the corresponding sample package picture, and the text description information is associated with an entity tag matching the corresponding sample package picture; input the text description information corresponding to each sample package picture into the large language model for rewriting, and obtain a plurality of rewritten text description information output by the large language model; use an image generation model to generate a plurality of text description information corresponding to each sample package picture according to the image generation model; Generate a test picture according to each rewritten text description information, and rewrite the corresponding rewritten text description information for the second time according to the content displayed by the test picture, and use the multiple text description information after the second rewriting as multiple first candidate text description information; use the commodity map model to construct multiple second candidate text description information, cross-recombination processing is performed on the multiple first candidate text description information and the multiple second candidate text description information to obtain multiple target text description information; construct multiple preset style layout templates based on the multiple target text description information, and use the entity label associated with each of the target text description information to annotate the generated preset style layout template;

[0160] Correspondingly, the determination module 302 is used to use the preset style layout template that matches the package text information among the multiple preset style layout templates as the at least one style layout template.

[0161] In a specific application scenario, the determination module 302 is used to determine the multiple sub-product names included in the sub-product set, determine the product information library corresponding to the store to which the package text information belongs in the server, and extract the product pictures corresponding to each of the sub-product names in the product information library; perform a completeness check on the product body in each of the extracted product pictures, and when it is detected that the completeness of a product body is lower than the completeness threshold, perform an extension operation on the product picture where the product body is located; perform a cutout operation on each of the product pictures to obtain a cutout result of each of the product pictures, and score the quality of the cutout result of each of the product pictures, and select a preset number of sub-product pictures for each of the sub-product names according to the obtained scoring results, to obtain the multiple sub-product pictures corresponding to the multiple sub-product names.

[0162] In a specific application scenario, the determination module 302 is used to compare the scoring result corresponding to each sub-product name with the result threshold; when the scoring result is greater than or equal to the result threshold, perform perspective detection on the product picture corresponding to the sub-product name, and select the preset number of sub-product pictures for the sub-product name based on the obtained perspective detection result; when the scoring result is less than the result threshold, determine the perspective priority, and in the preset product gallery, select the preset product pictures that match the sub-product name and are equal to the preset number as the preset number of sub-product pictures according to the perspective priority.

[0163] In a specific application scenario, the determination module 302 is used to search the preset product gallery for a specified preset product image that matches the sub-product name and the product image and has an oblique perspective when the perspective detection result indicates that the product image is a front view; if the search determines that the specified preset product image exists, the specified preset product image is used as a sub-product image, and in the preset product gallery, search for a first other preset product image that matches the sub-product name and the product image at each perspective according to the perspective priority, and use the first other preset product image and the specified preset product image that are searched for as the preset number of sub-product images; if the search determines that the specified preset product image does not exist, use the product image as a sub-product image, and in the preset product gallery, search for a second other preset product image that matches the sub-product name and the product image at each perspective according to the perspective priority, and use the second other preset product image and the product image that are searched for as the preset number of sub-product images.

[0164] In a specific application scenario, the determination module 302 is also used to, when the perspective detection result indicates that the product picture is not a front view, use the product picture as a sub-product picture, and search in the preset product picture library for a third other preset product picture that matches the sub-product name and the product picture at each perspective according to the perspective priority, and use the searched third other preset product picture and the product picture as the preset number of sub-product pictures; and / or, when multiple candidate preset product pictures that match the sub-product name and the product picture are searched for in the preset product picture library for a perspective, calculate the image similarity between each candidate preset product picture and the product picture, and use the candidate preset product picture with the largest image similarity among the multiple candidate preset product pictures as the other preset product picture that matches the sub-product name and the product picture searched at the perspective.

[0165] In a specific application scenario, the combination module 303 is used to determine the multiple sub-product names included in the sub-product set, and randomly select a sub-product picture for each of the multiple sub-product pictures; use each of the style layout templates to combine and layout the sub-product pictures currently selected for each of the sub-product names to obtain a meal package picture generated based on the currently selected sub-product picture; randomly select a new sub-product picture for each of the sub-product names and use each of the style layout templates to generate a meal package picture, until each of the multiple sub-product pictures is traversed to obtain and output at least one meal package picture.

[0166] In a specific application scenario, the combination module 303 is used to identify the preset layer information of the style layout template for each of the style layout templates; perform layer information initialization operations and layout initialization operations according to the preset layer information and the sub-product pictures currently selected for each of the sub-product names to obtain a layout framework; determine the mapping order indicated by the layout framework, select the first sub-product picture to be laid out from the sub-product pictures currently selected for each of the sub-product names according to the mapping order, and update the layer position information of the layer where the first sub-product picture is located within the layout framework according to the canvas zoom size indicated by the layout framework; continue to determine the second sub-product picture to be laid out from the sub-product pictures currently selected for each of the sub-product names according to the mapping order, determine the layer center of the layer where the second sub-product picture is located, and in the layout The position point to be laid out is determined on the straight line between the layout center indicated by the framework and the center of the layer, and the layer position information of the layer where the second sub-product picture is located in the layout framework is updated with reference to the position point to be laid out, and the third sub-product picture to be laid out indicated by the layout framework is continued to be determined in the sub-product picture currently selected for each of the sub-product names, and the position point to be laid out is determined for the third sub-product picture and the layer position information is updated until the layer position information update processing of the sub-product picture currently selected for each of the sub-product names is completed to obtain the layout framework to be used; the layers in the layout framework to be used are reordered according to the layer index, and the sub-product picture currently selected for each of the sub-product names is mapped with reference to the reordered layout framework to be used to obtain the set meal picture generated based on the currently selected sub-product picture.

[0167] In a specific application scenario, the combination module 303 is used to calculate layer parameters according to the preset layer information, scale the canvas to a preset boundary box, and use the layer parameters and the scaled canvas to construct an initial frame to complete the layer information initialization operation, wherein the layer parameters include the initial layout center, the main body area of ​​the main object in the layer, and the distance between each layer; refer to the sub-product illustration currently selected for each sub-product name to perform a canvas expansion operation on the initial frame, and adjust the initial layout center in the initial frame according to the expanded canvas to obtain the layout center, and refer to the position and distance between each layer indicated by the initial frame to determine the mapping order of the sub-product illustration currently selected for each sub-product name, complete the layout initialization operation, and obtain the layout frame.

[0168] In a specific application scenario, the combination module 303 is used to use the layout center as the left endpoint of the layer where the second sub-product illustration is located; determine a preset multiple, calculate the length of the line segment between the layout center and the layer center, and use the point on the straight line between the layout center and the layer center and the distance from the left endpoint to the preset multiple of the line segment length as the right endpoint of the layer where the second sub-product illustration is located; determine the middle position point according to the distance between the left endpoint and the right endpoint; query the specified layer with the middle position point as the layer center in the layout framework, and calculate the overlapping pixel area between the specified layer and the layer where the second sub-product illustration is located; if the overlapping pixel area If the product is not 0, the overlapping pixel area is used to calculate the overlapping ratio between the two layers, and the position point to be laid out is determined for the second sub-product illustration with reference to the overlapping ratio; if the overlapping pixel area is 0, the middle position point is used as the right endpoint of the layer where the second sub-product illustration is located, the left endpoint is re-determined using the updated right endpoint and the new middle position point is recalculated, and the overlapping pixel area is recalculated for the layer where the second sub-product illustration is located based on the new middle position point, until the calculated overlapping pixel area is not 0, the overlapping ratio between the two layers is calculated using the currently calculated overlapping pixel area, and the position point to be laid out is determined for the second sub-product illustration with reference to the overlapping ratio.

[0169] In a specific application scenario, the combination module 303 is used to obtain a ratio threshold and compare the overlap ratio with the ratio threshold; when the overlap ratio is less than the ratio threshold and greater than 0, the position point to be laid out is formed based on the left endpoint and the right endpoint; when the overlap ratio is greater than the ratio threshold, the middle position point is used as the left endpoint of the layer where the second sub-product illustration is located, the right endpoint is re-determined using the updated left endpoint and the new middle position point is recalculated, and the overlap ratio is recalculated for the layer where the second sub-product illustration is located based on the new middle position point and compared, until the calculated overlap ratio is less than the ratio threshold and greater than 0, and the position point to be laid out is formed based on the current left endpoint and the current right endpoint.

[0170] The device provided in the embodiment of the present application determines the package text information for which the illustration is to be generated, parses the package text information, obtains a sub-product set corresponding to the package text information, selects multiple sub-product illustrations based on the sub-product set, and determines at least one style layout template matching the package text information, combines and lays out the multiple sub-product illustrations with reference to the at least one style layout template, obtains and outputs at least one package illustration, and can automatically parse and determine the names of the sub-products included in the package text information and select an independent illustration for each sub-product name, and combines the independent illustrations using the style layout template matching the package text information to generate a complete package illustration. There is no need for merchants to illustrate the packages themselves, and while simplifying the illustration process for the packages and improving the illustration efficiency, the style layout template is used to ensure the consistency of the illustration style and the controllability of the quality, thereby improving the overall visual effect of the package illustration.

[0171] It should be noted that for other corresponding descriptions of the functional units involved in the set meal configuration image generation device provided in the embodiment of the present application, reference can be made to Figure 1 and FIG. 2A to FIG. 2I The corresponding description in will not be repeated here.

[0172] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0173] The above embodiments and the technical features in the embodiments can be arbitrarily combined with each other. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0174] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

[0175] In an exemplary embodiment, see Figure 4 , and also provides a computer device, which includes a bus, a processor, a memory and a communication interface, and may also include an input and output interface and a display device, wherein each functional unit can communicate with each other through the bus. The memory stores a computer program, and the processor is used to execute the program stored in the memory and execute the meal set map generation method in the above embodiment.

[0176] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the meal set image generation method.

[0177] Through the description of the above implementation methods, those skilled in the art can clearly understand that the present application can be implemented by hardware, or by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each implementation scenario of the present application.

[0178] Those skilled in the art will appreciate that the accompanying drawings are merely schematic diagrams of a preferred implementation scenario, and the modules or processes in the accompanying drawings are not necessarily required for implementing the present application.

[0179] Those skilled in the art will appreciate that the modules in the device in the implementation scenario can be distributed in the device in the implementation scenario according to the implementation scenario description, or can be changed accordingly and located in one or more devices different from the implementation scenario. The modules in the above implementation scenario can be combined into one module, or can be further split into multiple sub-modules.

[0180] The above application serial numbers are for description only and do not represent the advantages or disadvantages of the implementation scenarios.

[0181] The above disclosure only discloses several specific implementation scenarios of the present application. However, the present application is not limited thereto, and any changes that can be conceived by technicians in this field should fall within the scope of protection of the present application.

Claims

1. A method for generating a meal set picture, characterized in that: include: Determine the text information of the package for which the picture is to be generated, parse the text information of the package, and obtain a sub-product set corresponding to the text information of the package; Based on the sub-product set, multiple sub-product pictures are selected, and at least one style layout template matching the package text information is determined; The plurality of sub-product pictures are combined and laid out with reference to the at least one style layout template to obtain and output at least one set meal picture.

2. The method according to claim 1, characterized in that The step of determining the text information of the package for which the accompanying image is to be generated, parsing the text information of the package, and obtaining a sub-product set corresponding to the text information of the package includes: In response to a merchant's request for generating a picture, determining a product package name indicated by the request for generating a picture, and using text information associated with the product package name as the package text information for generating a picture; Using the product graph model to extract entity semantic information from the package text information, a plurality of sub-product names and product entity information corresponding to each of the sub-product names are obtained, wherein each of the product entity information is annotated with semantic information; Determine, in the server, a product information library corresponding to the store of the merchant, and obtain, in the product information library, a product image matching each of the sub-product names; Using a multimodal visual model to perform visual analysis on the product image corresponding to each of the sub-product names, to obtain the product attributes corresponding to each of the sub-product names; Constructing an initial product set based on the product attributes and product entity information corresponding to each of the sub-product names; The initial product set is output according to a predetermined structured format to obtain the sub-product set.

3. The method according to claim 2, characterized in that Before responding to the merchant's request for generating a picture, determining the name of the product package indicated by the request for generating a picture, and using the text information associated with the product package name as the package text information for generating a picture, the method further includes: Matching a plurality of sample product information, analyzing the product attributes corresponding to each of the sample product information, and constructing the product graph model based on the plurality of sample product information and the product attributes corresponding to each of the sample product information; A plurality of sample questions are set, an answer template corresponding to each of the sample questions is constructed, and the initial visual model is guided and trained using the plurality of sample questions and the answer template corresponding to each of the sample questions to obtain the multimodal visual model, wherein the plurality of sample questions include text questions and picture questions.

4. The method according to claim 3, characterized in that The matching of multiple sample product information and analyzing the product attributes corresponding to each sample product information includes: Matching the products connected to the platform where the store of the merchant is located, and using the product information of the matched multiple products as the multiple sample product information; Using a pre-trained deep language model, adaptively fine-tuning each sample product information; Using a two-layer bidirectional long short-term memory network to encode each of the sample product information that has undergone adaptive fine-tuning processing, to obtain text feature information corresponding to each of the sample product information; The text feature information of each sample product information is input into a conditional random field for parsing to obtain at least one entity tag corresponding to each sample product information, and the obtained at least one entity tag is used as a product attribute of the corresponding sample product information.

5. The method according to claim 4, characterized in that The method of using the product graph model to extract entity semantic information from the package text information to obtain multiple sub-product names and product entity information corresponding to each of the sub-product names includes: Analyze the package text information using the product graph model to obtain the multiple sub-product names; The following processing is performed on each of the sub-product names: determining matching target sample product information for the sub-product name from the plurality of sample product information, acquiring at least one first entity tag corresponding to the target sample product information, and determining characters to be analyzed in the sub-product name, acquiring at least one second entity tag matching the characters to be analyzed, and using the at least one first entity tag and the at least one second entity tag as product entity information corresponding to the sub-product name, wherein characters constituting the target product name of the target sample product information are included in the sub-product name, and the characters to be analyzed are other characters in the sub-product name except the characters constituting the target product name; Obtain the commodity entity information corresponding to each of the sub-commodity names.

6. The method according to claim 1, characterized in that Before selecting a plurality of sub-product pictures based on the sub-product set and determining at least one style layout template matching the package text information, the method further includes: Obtain multiple sample meal set pictures, and determine the design element information of each sample meal set picture; Using a multimodal visual model to process each of the sample package pictures and the corresponding design element information, to obtain text description information corresponding to each of the sample package pictures, wherein the text description information is used to describe the design style of the corresponding sample package pictures, and the text description information is associated with an entity tag matching the corresponding sample package pictures; Inputting the text description information corresponding to each of the sample set meal pictures into the large language model for rewriting, and obtaining a plurality of rewritten text description information output by the large language model; Generate a test image based on each rewritten text description information using an image generation model, rewrite the corresponding rewritten text description information according to the content displayed by the test image, and use the multiple text description information after the secondary rewriting as multiple first candidate text description information; Constructing a plurality of second candidate text description information using the product graph model, and cross-combining the plurality of first candidate text description information and the plurality of second candidate text description information to obtain a plurality of target text description information; Constructing a plurality of preset style layout templates based on the plurality of target text description information, and annotating the generated preset style layout templates with entity tags associated with each of the target text description information; Accordingly, the determining of at least one style layout template matching the package text information includes: A preset style layout template among the multiple preset style layout templates that matches the package text information is used as the at least one style layout template.

7. The method according to claim 1, characterized in that The selecting a plurality of sub-product pictures based on the sub-product set includes: Determine multiple sub-product names included in the sub-product set, determine a product information library corresponding to the store to which the package text information belongs in the server, and extract a product picture corresponding to each sub-product name in the product information library; Performing a completeness check on the product body in each of the extracted product pictures, and when it is detected that the completeness of a product body is lower than a completeness threshold, performing an extension operation on the product picture where the product body is located; A cutout operation is performed on each of the product pictures to obtain a cutout result of each of the product pictures, and the quality of the cutout result of each of the product pictures is scored; and based on the obtained scoring results, a preset number of sub-product pictures are selected for each of the sub-product names to obtain the multiple sub-product pictures corresponding to the multiple sub-product names.

8. A set meal picture generating device, characterized in that: include: A parsing module, used to determine the text information of the package for which the accompanying image is to be generated, parse the text information of the package, and obtain a sub-product set corresponding to the text information of the package; A determination module, configured to select a plurality of sub-product pictures based on the sub-product set, and determine at least one style layout template matching the package text information; The combination module is used to combine and layout the multiple sub-product pictures with reference to the at least one style layout template, obtain at least one set meal picture and output it.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images