Packaging design methods and apparatus, electronic equipment and storage media
By building an integrated intelligent design workflow, and utilizing image generation models, large language models, and mixed reality rendering engines, the problem of fragmentation in various stages of packaging design has been solved, achieving an efficient and intelligent packaging design process and intuitive 3D verification.
Patent Information
- Application Number
- CN202611051842.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-15
- Publication Date
- 2026-11-17
AI Technical Summary
The existing packaging design process is fragmented, with low levels of intelligence, a lack of visual evaluation and feedback loops, and difficulty in achieving three-dimensional virtual-real mapping, resulting in low design iteration efficiency and resource waste.
We construct an integrated intelligent design workflow, which uses image generation models, large language models and mixed reality rendering engines to achieve iterative processing from sketch data to 3D preview. Combined with multimodal interaction and visual evaluation, it provides optimization feedback and virtual-real mapping.
It improves the intelligence and continuity of the packaging design process, lowers the design threshold, increases the efficiency and accuracy of design iteration, reduces trial and error costs, and enables intuitive verification of design solutions in real-world environments.
Smart Images

Figure CN123116772A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of artificial intelligence technology, and in particular to a packaging design method and apparatus, electronic device and storage medium. Background Technology
[0002] The development of Internet technology has driven the development of artificial intelligence technology, which in turn has promoted the prosperity of Artificial Intelligence Generated Content (AIGC). AIGC has shown great application value in many physical manufacturing and creative industries. For example, in the packaging design scenario, AIGC technology has been widely used in the packaging concept drawing generation stage due to its efficient image generation capabilities.
[0003] However, in practice, related technologies typically use AIGC as a one-way image generation tool. The packaging design workflow in these technologies relies on multiple independent software systems to perform separate processes such as creative generation, 2D sketching, and rendering preview. They cannot provide intelligent modification guidance for the generated design drafts and still heavily rely on the designer's subjective experience for manual blind testing and tedious trial and error with prompts. Furthermore, they cannot intuitively show the designer the presentation of the packaging design on the packaging carrier.
[0004] Therefore, the generative packaging design workflow in related technologies suffers from low intelligence levels in the overall packaging design process due to the fragmentation of each operational step and the low level of intelligence in each step. Summary of the Invention
[0005] The main objective of this application is to propose a packaging design method and apparatus, electronic device and storage medium, which aims to build an integrated intelligent design workflow, improve the continuity of the entire packaging design process, and thus enhance the intelligence level of the packaging design process.
[0006] To achieve the above objectives, a first aspect of this application provides a packaging design method, the method comprising: The system acquires packaging design sketch data sent by the client and prompt words used to render and describe the packaging design sketch data, and calls a preset image generation model to generate an initial packaging design image based on the packaging design sketch data and the prompt words data. The system calls a preset large language model to evaluate the initial packaging design image, generates corresponding optimization feedback data based on the evaluation results, and sends the optimization feedback data to the client. The system receives an image editing and optimization instruction from the client, which is based on the optimization feedback data, and performs an image update operation on the initial packaging design image according to the image editing and optimization instruction to obtain an updated packaging design image. The carrier image data corresponding to the target packaging carrier is obtained, and a preset mixed reality rendering engine is called to perform a three-dimensional preview operation based on the updated packaging design image, the carrier image data, and the target virtual three-dimensional model corresponding to the target packaging carrier, and generate the corresponding three-dimensional preview result. The updated packaging design image is used as the new initial packaging design image. At least one round of iterative processing is performed based on at least one of the evaluation operation, the image update operation, and the 3D preview operation until the final confirmation instruction sent by the client is received. The target packaging design image is then determined based on the final updated packaging design image.
[0007] To achieve the above objectives, a second aspect of this application provides a packaging design apparatus, the apparatus comprising: The first generation unit is used to acquire packaging design sketch data sent by the client and prompt word data for rendering description of the packaging design sketch data, and call a preset image generation model to generate an initial packaging design image based on the packaging design sketch data and the prompt word data. The sending unit is used to call a preset large language model to evaluate the initial packaging design image, generate corresponding optimization feedback data based on the evaluation results, and send the optimization feedback data to the client. The update unit is used to receive the image editing optimization instruction sent by the client based on the optimization feedback data, and to perform an image update operation on the initial packaging design image according to the image editing optimization instruction to obtain an updated packaging design image; The second generation unit is used to acquire the carrier image data corresponding to the target packaging carrier, and call the preset mixed reality rendering engine to perform a three-dimensional preview operation based on the updated packaging design image, the carrier image data and the target virtual three-dimensional model corresponding to the target packaging carrier, and generate the corresponding three-dimensional preview result. The determining unit is used to take the updated packaging design image as the new initial packaging design image, and perform at least one round of iterative processing based on at least one of the evaluation operation, the image update operation, and the 3D preview operation until the final confirmation instruction sent by the client is received, and determine the target packaging design image based on the final updated packaging design image.
[0008] To achieve the above objectives, a third aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect.
[0009] To achieve the above objectives, a fourth aspect of the present application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in the first aspect.
[0010] The packaging design method, apparatus, electronic device, and storage medium proposed in this application generate an initial packaging design image based on packaging design sketch data and prompt word data by calling an image generation model, thus reducing the design threshold and trial-and-error costs in the early stages. Subsequently, a large language model is called to evaluate the initial image and output optimization feedback data, overcoming the shortcomings of existing technologies such as the lack of visual evaluation loop and reliance on blind manual testing. Furthermore, by receiving image editing and optimization instructions from the client and performing image update operations to obtain an updated packaging design image, collaborative modification with human-led and AI-assisted modification is achieved, improving the accuracy and efficiency of design modification. Next, a mixed reality rendering engine is called to perform 3D preview operations in combination with the target virtual 3D model, solving the problems of traditional design schemes lacking spatial verification capabilities and difficulty in intuitively examining the actual mapping effect. Finally, the updated packaging design image undergoes at least one round of iterative processing based on evaluation, update, and 3D preview until a final confirmation instruction is received. In this way, this application constructs an integrated intelligent packaging design workflow, improves the coherence of the entire packaging design chain, and thus enhances the intelligence level of the packaging design process. Attached Figure Description
[0011] Figure 1 This is a flowchart of the packaging design method provided in the embodiments of this application; Figure 2 yes Figure 1 The flowchart of step S101 in the text; Figure 3 yes Figure 1 Another flowchart of step S101 in the process; Figure 4 This is a schematic diagram of the packaging design interface provided in this application; Figure 5 yes Figure 1 The flowchart of step S104 in the process; Figure 6 yes Figure 5 The flowchart of step S501 in the process; Figure 7 This is another flowchart of the packaging design method provided in this application; Figure 8 This is an architecture diagram of the packaging design system provided in this application; Figure 9 This is another flowchart of the packaging design method provided in this application; Figure 10 This is a schematic diagram of the packaging design device provided in the embodiments of this application; Figure 11 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation
[0012] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0013] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.
[0014] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.
[0015] The development of internet technology has driven the advancement of artificial intelligence (AI) technology, fostering the prosperity of AI-generated content (AIGC). AIGC has demonstrated significant application value in numerous manufacturing and creative industries, particularly in packaging design. AIGC technology, with its efficient image generation capabilities, has been widely used in concept creation and early exploration of design styles.
[0016] However, in practice, related technologies typically only use AIGC as a one-way two-dimensional image generation tool. Existing packaging design technologies and workflows still generally suffer from numerous technical deficiencies in practical applications. Specifically, packaging design workflows in these technologies often rely on multiple independent software systems to perform fragmented processes such as idea generation, 2D sketching, 3D modeling, and rendering preview. This cross-software operation mode not only leads to cumbersome data format conversion and high costs for manual intervention, making it difficult to form a coherent and integrated automated design workflow, but also results in a very weak version management mechanism for multi-dimensional data.
[0017] Because packaging design typically requires multiple rounds of high-frequency iterations, existing systems lack a unified and interconnected management mechanism for the complete data chain of packaging design iterations. This makes it difficult to effectively trace, compare, and reuse historical version data from each iteration stage, ultimately resulting in a serious waste of design assets.
[0018] Furthermore, existing systems suffer from a limited human-computer interaction model and a severe lack of visual evaluation and feedback loops. Current AIGC design tools rely excessively on precise text prompts, posing a high barrier to entry for designers lacking experience in prompt engineering.
[0019] Meanwhile, existing design iterations are mostly offline or one-way operation modes, lacking continuous interaction mechanisms based on multimodal approaches such as natural language, making it difficult to achieve real-time creative modifications based on ideas. After generating images, existing technologies cannot provide intelligent modification guidance for the generated design drafts, and visual evaluation of the generated images is difficult to form an automated closed-loop feedback with parameter adjustments of the generated model.
[0020] After receiving the review results, designers still heavily rely on subjective experience for manual blind testing and tedious trial-and-error using prompts. This leads to a severe disconnect between the review and revision processes, easily resulting in an inefficient iterative cycle. Finally, existing technologies lack a 3D virtual-real mapping mechanism, resulting in weak spatial verification capabilities. Current systems typically only support previewing and evaluating design schemes on 2D display terminals, failing to intuitively show designers how packaging designs appear on real packaging carriers. Furthermore, they cannot map the generated 2D designs as texture maps onto physical packaging entities (such as boxes or bottles) in real time and with precision. This prevents designers from intuitively and accurately verifying the visual effects, proportions, and physical consistency of their designs in real 3D physical space early in the design process.
[0021] In summary, the generative packaging design workflow in related technologies suffers from a severe lack of intelligence and coherence due to the fragmentation of each operational step and the low level of intelligence in each step.
[0022] Based on this, embodiments of this application provide a packaging design method and apparatus, electronic device and storage medium, aiming to build an integrated intelligent design workflow, improve the continuity of the entire packaging design chain, and thereby improve the intelligence level of the packaging design process.
[0023] The packaging design method, apparatus, electronic device, and storage medium provided in this application are specifically described through the following embodiments. First, the packaging design method in this application is described.
[0024] The packaging design method provided in this application relates to the field of artificial intelligence technology. The packaging design method provided in this application can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application implementing the packaging design method, but is not limited to the above forms.
[0025] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.
[0026] Figure 1 This is an optional flowchart of the packaging design method provided in the embodiments of this application. Figure 1 The method may include, but is not limited to, steps S101 to S105.
[0027] Step S101: Obtain the packaging design sketch data sent by the client and the prompt words data used to render the packaging design sketch data, and call the preset image generation model to generate an initial packaging design image based on the packaging design sketch data and the prompt words data; Step S102: Call the preset large language model to evaluate the initial packaging design image, generate corresponding optimization feedback data based on the evaluation results, and send the optimization feedback data to the client. Step S103: Receive the image editing optimization instruction sent by the client based on the optimization feedback data, and perform an image update operation on the initial packaging design image according to the image editing optimization instruction to obtain the updated packaging design image; Step S104: Obtain the carrier image data corresponding to the target packaging carrier, and call the preset mixed reality rendering engine to perform a three-dimensional preview operation based on the updated packaging design image, carrier image data and the target virtual three-dimensional model corresponding to the target packaging carrier, and generate the corresponding three-dimensional preview result. Step S105: The updated packaging design image is used as the new initial packaging design image. At least one round of iterative processing is performed based on at least one of the evaluation operation, image update operation, and 3D preview operation until the final confirmation instruction sent by the client is received. The target packaging design image is determined based on the final updated packaging design image.
[0028] The packaging design method described in this application is applied to the server side. The server receives multimodal input data sent from the client, calls a real-time conversational AI based on a large language model deployed on the backend to perform semantic parsing and intent extraction, and sends the packaged data to the backend image generation service node to perform high-computing-power model inference calculations to generate initial or updated 2D images of the packaging design. Simultaneously, the server-side task scheduling and result management module calls a visual analysis service to perform feature extraction and design rule evaluation on the generated 2D images, and establishes a mapping relationship between multi-dimensional data such as sketches, prompts, optimization feedback, and generated images at the underlying level to generate independent and traceable historical packaging design version data. Finally, the server sends back the generated heuristic suggestions, optimization feedback data including text and voice, and corresponding version thumbnails and other digital asset processing results to the client in real time for the frontend to perform mixed reality virtual-real mapping rendering and interactive interface display. By centrally deploying resource-intensive AI generation and visual understanding tasks on the server side, this application can effectively overcome the hardware computing power bottleneck of front-end application terminal devices, ensure data processing efficiency and model delivery response speed, and provide underlying support for realizing low-latency instant creative modification, seamless integrated workflow, and unified version management of historical design assets.
[0029] In step S101, packaging design sketch data refers to the two-dimensional outline information drawn by the user on a canvas using an input device (such as a drawing tool). For example, when designing juice packaging, the user draws an orange outline and a curved sketch representing flowing juice. Prompt data represents descriptive text or speech input by the user to describe details of the packaging image, such as design style, elements, and color preferences. For example, a modern minimalist orange juice packaging with bright orange and white as the main colors and a splashing juice effect. Prompt data can be collected in natural language speech form through a multimodal interactive interface. For example, the user inputs voice commands to the client through an audio input device. The client performs speech recognition and semantic parsing on the voice commands, converting them into text prompt data describing the design style, elements, and colors, thus obtaining prompt information without relying on traditional keyboard input methods.
[0030] The client packages the packaging design sketch data and prompt text data and sends them to the server. The server then calls a pre-defined image generation model to generate an initial packaging design image based on the sketch and prompt text data. This image generation model can be a deep learning model trained on a diffusion model or a generative adversarial network architecture; there are no restrictions here. The packaging design sketch data provides visual guidance information for the image generation model, while the prompt text data provides semantic description information. This embodiment breaks away from the excessive reliance of traditional AIGC tools on precise text prompts, allowing users to interact through a more intuitive multimodal approach (sketches plus natural language), reducing the barrier to entry and initial trial-and-error costs for designers without a technical background.
[0031] In some embodiments, please refer to Figure 2 Obtain the packaging design sketch data sent by the client, including: Step S201: Receive natural language instructions sent by the client, and perform semantic analysis on the natural language instructions to obtain the packaging design intent; Step S202: Generate packaging design inspiration suggestions based on the packaging design intent, and obtain the packaging design sketch data obtained by the client based on the packaging design inspiration suggestions.
[0032] In step S201, the client is equipped with a real-time conversational AI module that possesses visual understanding capabilities and is based on a Large Language Model (LLM). This module can receive natural language command data from the user and then send the data to the server. The natural language commands can be voice commands transmitted by the user through audio input devices such as microphones, or text input commands that express initial design concepts and interaction needs. For example, a user might input, "I want a juice packaging suitable for young consumers, with a fresh and natural style." Upon receiving the natural language command, the server can invoke the LLM to perform intent recognition, entity extraction, and sentiment analysis. By parsing keywords, sentence structures, and contextual relationships within the natural language command, the server extracts design requirement information from dimensions such as packaging type, target audience, style preferences, and emotional appeals. This extracted design requirement information is then integrated to obtain a structured packaging design intent.
[0033] In step S202, the large language model is then invoked to generate packaging design inspiration suggestions based on the packaging design intent. These suggestions provide specific guidance for users in their subsequent design creation, including layout principles, color matching strategies, and the presentation of brand elements. The server can provide these suggestions to the client via real-time voice and text replies, thus assisting users in establishing initial design directions and preliminary design concepts. For example, in the application scenario of designing a new juice box packaging, when a user expresses a design intent targeting a young audience and requiring a modern, minimalist, and vibrant style, the conversational AI module can not only accurately interpret this intent but also provide packaging design inspiration suggestions such as "We recommend using sans-serif fonts and color block composition" or "Consider using highly saturated orange-yellow tones."
[0034] After receiving packaging design inspiration suggestions, users read, filter, and confirm them to determine the initial design direction for drawing packaging design sketches. The initial design direction includes the overall style and tone of the packaging design, the color scheme framework, and the layout planning of visual elements. Furthermore, users can use interactive input devices (such as digital paintbrushes) to draw packaging sketches on a canvas within the client's packaging design interface, obtain packaging design sketch data, and send the packaging design sketch data to the server.
[0035] This embodiment reduces the reliance of traditional AI design tools on complex prompts by constructing a coherent interactive chain from natural language intent understanding to design inspiration guidance and sketch data acquisition. This reduces the cost of initial blind trial and error and directional exploration, allowing users to instantly obtain accurate inspirational suggestions through intuitive voice dialogue and sketching. Thus, this embodiment eliminates data conversion steps caused by cross-platform switching, improving efficiency in the initial stages of packaging design, the intuitiveness of intent expression, and the overall quality of subsequent creative iterations. It also enhances the intelligence and user experience of the front-end interactive links in the packaging design process.
[0036] In some embodiments, please refer to Figure 3 The system calls a preset image generation model to generate an initial packaging design image based on the packaging design sketch data and prompt word data, including: Step S301: Obtain background image data corresponding to the display background of the target packaging carrier, and extract features from the background image data to obtain the corresponding three-dimensional background environment features. Step S302: Map the 3D background environment features to the corresponding environmental constraint semantic labels, and generate fused prompt word data based on the environmental constraint semantic labels and prompt word data; Step S303: Call the preset image generation model to generate an initial packaging design image based on the packaging design sketch data and the fused prompt word data.
[0037] In step S301, background image data corresponding to the display background of the target packaging carrier is acquired. In the actual design workflow, the target packaging carrier needs to be placed in a specific environment for display and evaluation. For example, packaging boxes or bottles are usually placed on supermarket shelves, wooden dining tables, or outdoor family dining tables. The client can use image acquisition devices to acquire real-time images of the actual environment in which the physical packaging is about to be placed or is currently placed, thus obtaining background image data.
[0038] The client sends background image data to the server, which then extracts features from the background image data. This extraction can be achieved using computer vision algorithms to analyze the background image data and obtain 3D background environmental features that represent the real-world environment. These features can include the depth structure of the background, the direction and color temperature of the light source, the dominant color of the background, and information about the materials used (such as wood grain, metal, and fabric). For example, when the target packaging is a juice box and the background is a supermarket shelf, the server can extract features such as the horizontal structure of the shelf shelves, the arrangement of competing product packaging, and the cool white light illumination from the top fluorescent lights.
[0039] In step S302, the environmental constraint semantic label represents an environmental attribute identifier described in natural language text form after semantic mapping processing of the 3D background environment features. This label is used to transform abstract visual features into semantic constraints that the image generation model can understand. For example, if the 3D background environment features show that the current display background lighting is warm and has strong diffuse reflection, and the background has a wood texture, then this is mapped to semantic text labels such as warm-toned lighting, diffuse environment, and wood background. Then, the environmental constraint semantic labels are concatenated with the prompt word data to generate fused prompt word data. In this way, this application can embed packaging display environment constraints into the packaging design intent, enabling the image generation model to balance user creative needs with the visual harmony of the actual display environment when generating the initial packaging design image.
[0040] In step S303, the server invokes the image generation model to generate an initial packaging design image based on the packaging design sketch data and the fused prompt data. During the inference process of the image generation model, the packaging design sketch data serves as a geometric constraint to control the overall layout and element outlines of the generated image, while the fused prompt data serves as a semantic constraint to control the style attributes, color distribution, and environmental adaptability of the generated image. Guided by the environmental constraint semantic tags, the image generation model can automatically supplement the generated image with information such as light and shadow distribution, color mapping, and material reflection that match the real display background.
[0041] This embodiment addresses the technical problem of traditional AIGC design tools creating images that are disconnected from the actual display environment. By extracting the 3D environmental features of the real display background and converting them into environmental constraint semantic tags, it effectively guides the image generation model to fully consider the packaging display background. This ensures that the generated packaging design images meet the user's creative needs while adapting to the real display scenario. Consequently, when the packaging design images are subsequently mapped onto physical packaging, they can seamlessly integrate with the real environment, reducing the need for repeated trial and error and redrawing due to visual inconsistencies in the scene. This improves the evaluation efficiency and quality of intelligent packaging design solutions.
[0042] In step S102, after the server generates the initial packaging design image, it calls a large language model to evaluate the initial packaging design image. This evaluation includes extracting and analyzing visual features from the initial packaging design image, and comparing the extracted visual features with preset design rules. These design rules include specifications for professional design dimensions such as composition rationality, color harmony, and brand element consistency. Specifically, the composition rationality rule evaluates whether the spatial distribution of visual elements in the image is balanced; the color harmony rule analyzes whether the matching relationship between the main color and auxiliary colors conforms to color theory principles; and the brand element consistency rule verifies whether the fonts, graphics, and logos in the image meet the requirements of the target brand.
[0043] After rule comparison, the large language model can generate optimization feedback data. This data represents specific modification suggestions for visual defects or areas for improvement in the current packaging design image. These suggestions can include descriptions of visual defects, composition adjustment suggestions, color matching optimization schemes, and brand element coordination tips—providing professional design guidance. The server can then send the optimization feedback data to the client in voice or text format. For example, the client can provide voice feedback: "The current image has saturated colors; try adding subtle dark textures to the blank areas to increase the image's depth." In this way, this embodiment avoids designers relying on subjective experience, blindly guessing, and repeatedly modifying, improving the efficiency of packaging design image evaluation and iterative updates.
[0044] In some embodiments, a preset large language model is invoked to evaluate the initial packaging design image, and corresponding optimization feedback data is generated based on the evaluation results, including: The preset visual saliency prediction model is invoked to generate a corresponding visual attention heatmap based on the initial packaging design image; Identify target design elements in the initial packaging design images and calculate the attention overlap of the target design elements in the visual attention heatmap; When the attention overlap is lower than the preset threshold, the visual attention heatmap and the spatial distribution information of the target design elements are used as constraints. The preset large language model is called to evaluate the initial packaging design image, and corresponding optimization feedback data is generated based on the evaluation results.
[0045] In this embodiment, the visual saliency prediction model is a model trained on a deep learning architecture that can simulate the human visual attention mechanism and predict the degree of attention received by each region in an image, such as the SALICON model or the DeepGaze series of models, etc., without limitation. The visual saliency prediction model can analyze visual features such as color contrast, edge strength, texture complexity, and semantic content of an image to output a probability distribution map corresponding to the spatial resolution of the input image. This allows it to predict the region where an observer's gaze is most likely to linger and concentrate when they first see the image, generating a corresponding visual attention heatmap. The visual attention heatmap can visually identify the strength of visual appeal of each region in the image using different color gradients (e.g., a gradient from cool to warm tones). For example, red or bright areas represent areas where visual focus is concentrated, while blue or dark areas represent secondary background that is easily ignored.
[0046] The server can then identify target design elements from the initial packaging design image. These target design elements represent the objects that the designer expects users to focus on first in the packaging design, such as the brand logo, the main visual image of the product (e.g., splashed juice or two birds perched on a branch in the image), or visual components such as selling point text. These elements can be identified through the user's selection or semantic annotation operations in the packaging design interface.
[0047] Furthermore, the server can calculate the attention overlap of the target design element in the visual attention heatmap, where attention overlap measures the degree of spatial overlap between the pixel region where the target design element is located and the highly salient regions (e.g., highlighted regions) in the heatmap. Specifically, the server overlays the outline mask or bounding box of the target design element onto the visual attention heatmap, and calculates the maximum, average, or integral of the heatmap values within the masked region to determine the actual visual attention allocated to the target design element.
[0048] High attention overlap indicates that the target design element has successfully attracted visual focus; conversely, low overlap indicates a weak visual presence. The server compares the attention overlap with a preset threshold, which can be adjusted based on different packaging design categories or brand strategy requirements. When the attention overlap is below the preset threshold, it indicates a visual communication defect in the current packaging design, meaning the target design element that the user expects to highlight has failed to effectively capture visual focus, and the viewer's attention may be distracted by other secondary elements. At this point, the server obtains the spatial distribution information of the target design element. This spatial distribution information represents the position range and geometric boundary information of the target design element in the initial packaging design image, which can be represented by pixel masks, bounding rectangle coordinate parameters, or boundary point sets. Then, based on the visual attention heatmap, the server analyzes the actual attention-attracting areas and intensity distribution in the initial packaging design image. Using the visual attention heatmap and the spatial distribution information of the target design element as constraints, the server calls a large language model to evaluate the initial packaging design image.
[0049] In this process, the large language model not only relies on design rules but also uses heatmap distribution data and the actual positions of target design elements as contextual parameters to analyze the initial packaging design image. After determining that there is a visual offset in the target design element and finding the cause of the visual offset, the large language model can generate corresponding optimization feedback data based on the evaluation results. For example, the output voice suggestion is: "The current brand logo area lacks visual appeal. It is recommended to reduce the contrast of the surrounding background texture, or increase the white space around the logo and improve its color saturation."
[0050] This embodiment addresses the technical problem of relying on subjective personal experience and the difficulty in quantifying guidance in visual reviews of packaging design. By introducing a visual saliency prediction model to generate heatmaps and quantifying the attentional overlap of core design elements, it accurately grasps the effectiveness of packaging design schemes in real-world visual communication, thereby enhancing the efficiency and attractiveness of information delivery in actual scenarios. When visual focus is found to be substandard, this embodiment provides targeted optimization strategies through a large language model, improving the professionalism and accuracy of design feedback and effectively guiding designers to make modifications.
[0051] In some embodiments, the process of displaying optimization feedback data on the client includes the following steps: The packaging design interface is displayed, and the packaging design interface includes an evaluation feedback area; The evaluation feedback area displays the optimization feedback data and provides voice prompts for the optimization feedback data.
[0052] In this embodiment, the packaging design interface is a graphical interface used for user interaction with the intelligent packaging design system, aiming to solve the technical problem of traditional design processes relying on multiple independent software systems for cross-platform operation. This embodiment divides the packaging design interface into multiple functional areas, including an evaluation feedback area that displays AI visual analysis results, text prompts for input feedback, and natural language interaction records. Thus, when users are sketching or making partial modifications to packaging designs, they can intuitively view the evaluation status of the current design scheme, reducing the costs of data format conversion and software switching.
[0053] Specifically, the server sends optimization feedback data to the client, which can render and display the optimization feedback data in the form of structured text within the evaluation feedback area. For example, it can display suggested text line by line, highlight key modification points, or present multiple optimization suggestions in a structured list, so that users can read and understand the design improvement opinions intuitively. Alternatively, the optimization feedback data can be broadcast synchronously via audio devices, achieving multimodal synchronous presentation of visual text and auditory speech.
[0054] This embodiment, through an integrated interface layout and a multimodal dual-channel feedback mechanism, displays an evaluation feedback area in the packaging design interface. Combined with text display and synchronous voice broadcast output, it overcomes the technical problem of the disconnect between the visual review of generated images and parameter modification in existing technologies. It avoids the problem of designers relying solely on subjective blind testing and repeatedly modifying prompts, which leads to low design efficiency. This improves the human-computer interaction efficiency and information transmission effect of the intelligent packaging design system, while also improving the iteration quality and production efficiency of packaging design solutions. Ultimately, it enhances the intelligence level and user experience of the review and feedback process in the packaging design workflow.
[0055] In some embodiments, please refer to Figure 4 , Figure 4 This is a schematic diagram of the packaging design interface provided in this application, such as... Figure 4 As shown, the packaging design interface 400 includes an output preview area 410, a central main view area 420, a tools and operation area 430, an AI interaction area 440, and a historical version management area 450. The tools and operation area 430 includes an operation area 431 and a tool area 432. Specifically, the output preview area 410 integrates an enable control for MR virtual-real mapping preview and a thumbnail of the 3D preview. The central main view area 420 is used to display the rendering effects of 2D sketch editing and texture mapping in real time. The operation area 431 provides image editing tools such as brushes and erasers. The tool area 432 provides creative tools such as partial redrawing, a color palette, and input prompts for texture generation. The AI interaction area 440 can provide feedback to the user in the form of text, audio, or animation after real-time conversational AI performs visual evaluation. The historical version management area 450 provides historical version thumbnails (e.g., version thumbnails). Figure 1 Version abbreviation Figure 2 It achieves unified association of multi-dimensional data links, allowing designers to switch and continue editing at any time by clicking.
[0056] In step S103, the image editing optimization instruction refers to the optimization instruction generated by the client based on the user's editing operations on the initial packaging design image on the packaging design interface after the user reads the optimization feedback data. The image editing optimization instruction may include instructions such as partial redrawing of image areas, adjustment of color parameters, and addition and affine transformation of graphic elements (e.g., importing a pre-saved brand logo as a sticker onto the canvas and customizing its size, position, and rotation angle). Then, the server can update the initial packaging design image according to the image editing optimization instruction, generating an updated packaging design image that better meets the creative requirements.
[0057] In step S104, video stream data of the target packaging carrier can be acquired in real time through an image acquisition device to obtain carrier image data. The target packaging carrier is a physical entity used to carry the final packaging design scheme, such as a physical packaging box, bottle, or can. The carrier image data contains visual information of the target packaging carrier in real three-dimensional space.
[0058] Then, a pre-built mixed reality rendering engine can be invoked to spatially align the pre-built virtual 3D model of the target packaging carrier with the real packaging model based on the real-time pose data of the target packaging carrier. The virtual 3D model of the target is a digital 3D model that corresponds to the geometry and size ratio of the target packaging carrier.
[0059] Furthermore, the updated packaging design image can be rendered in real time and overlaid onto the target virtual 3D model to generate a 3D preview result. This preview allows users to freely rotate and move the target packaging carrier, intuitively examining the visual effect, proportions, and harmony of the design scheme in real 3D space from any perspective. Thus, this embodiment, through the aforementioned virtual-real mapping mechanism, can intuitively and accurately verify the display effect of the design pattern in a real environment in a WYSIWYG manner.
[0060] In some embodiments, please refer to Figure 5 The system invokes a preset mixed reality rendering engine to perform a 3D preview operation based on the updated packaging design image, carrier image data, and the target virtual 3D model corresponding to the target packaging carrier, generating the corresponding 3D preview results, including: Step S501: Identify the target positioning feature information corresponding to the preset mark on the surface of the target packaging carrier in the carrier image data, and calculate the pose data of the target packaging carrier in the real three-dimensional space based on the target positioning feature information. Step S502: The preset mixed reality rendering engine is invoked to spatially align the target packaging carrier and the corresponding target virtual 3D model based on the pose data, and the updated packaging design image is mapped as a texture map onto the surface of the spatially aligned target virtual 3D model to generate the corresponding 3D preview result.
[0061] In step S501, the client can acquire carrier image data containing the target packaging carrier in real time through an image acquisition device, and then identify the target positioning feature information corresponding to preset marks on the surface of the target packaging carrier in the carrier image data. The preset marks are identifiers with specific geometric shapes or patterns pre-set on the surface of the target packaging carrier, used to provide traceable spatial reference points. The preset marks can be high-contrast visual recognition marks similar to ArUco reference codes, whose corner points and internal features are easily extracted and matched quickly in complex image backgrounds. Multiple preset marks can be present on the target packaging carrier. In actual mixed reality interaction, users need to freely rotate the target packaging carrier; a single mark can easily move out of the camera's field of view due to viewpoint rotation or flipping. Using multiple preset marks distributed on different surfaces of the target packaging carrier can improve the continuity of pose tracking in three-dimensional space. The target positioning feature information is information extracted from the carrier image data that characterizes the geometric attributes of the preset marks, such as their position, outline, and corner coordinates.
[0062] Then, the server can use computer vision algorithms to perform inverse geometric calculations on the 3D perspective deformation of the preset markers, thereby calculating the pose data of the target packaging carrier in real 3D space based on the target positioning feature information. The pose data includes the position of the target packaging carrier in the real 3D coordinate system (i.e., translation, such as X, Y, and Z axis coordinates) and spatial orientation (i.e., rotation, such as pitch, yaw, and roll angles), which can be represented by rotation matrices or quaternions. In this way, the server can perceive the current position and attitude changes of the target packaging carrier in real time and accurately.
[0063] In some embodiments, the carrier image data further includes image data that simultaneously includes the target packaging carrier and the operating components for operating the target packaging carrier. Please refer to... Figure 6 The method involves identifying target positioning feature information corresponding to preset marks on the surface of the target packaging carrier in the carrier image data, and calculating the pose data of the target packaging carrier in real three-dimensional space based on the target positioning feature information, including: Step S601: Extract feature point location information of the operating component from the carrier image data; Step S602: When the target positioning feature information corresponding to the preset mark on the surface of the target packaging carrier is successfully identified in the carrier image data, the pose data of the target packaging carrier in the real three-dimensional space is calculated based on the target positioning feature information, and the relative spatial transformation relationship between the feature point position information and the calculated pose data is determined. Step S603: When the recognition of target positioning feature information fails, pose inverse compensation is performed based on the relative spatial transformation relationship and feature point position information to obtain the pose data of the target packaging carrier in real three-dimensional space.
[0064] In this embodiment, the carrier image data not only includes visual information of the target packaging carrier itself, but also image data of the operating components used to operate the target packaging carrier; wherein, the operating components refer to the human body parts or handheld tools used by the user to hold, flip or move the target packaging carrier in a real interactive scenario, such as the user's hand or auxiliary gripping device.
[0065] In the actual mixed reality preview process, the target packaging carrier needs to be freely rotated and repositioned from multiple angles. When the image acquisition device captures the target packaging carrier in real time, its field of view will inevitably include the operating parts in the acquisition screen. Therefore, the carrier image data actually presents the target packaging carrier and the operating parts in the same image frame.
[0066] In step S601, feature point location information of the operating component is extracted from the carrier image data. Specifically, target detection model or skeletal key point tracking technology can be used to identify the edge contour feature points of the operating component in consecutive image frames, such as the fingertip position and the three-dimensional spatial coordinates of the joint node. These coordinate data constitute the feature point location information.
[0067] In step S602, when the server successfully identifies the target positioning feature information corresponding to the preset mark on the surface of the target packaging carrier in the carrier image data, the server not only calculates the pose data of the target packaging carrier in the real three-dimensional space based on the target positioning feature information, but also performs spatial association operations. Specifically, the feature point position information of the operating component within the same frame is extracted, and the relative spatial transformation relationship between the feature point position information and the calculated pose data of the target packaging carrier in the three-dimensional space is determined through spatial geometric matrix operations. This can include the distance deviation and rotation angle offset between the key points of the operating component and the center coordinate system of the target packaging carrier when the operating component grasps the target packaging carrier. The relative spatial transformation relationship can be described by translation vectors and rotation matrices. The relative spatial transformation relationship can be used for pose data compensation calculation when the preset mark is subsequently occluded.
[0068] In step S603, when a user flips or moves the target packaging carrier in a mixed reality environment to observe the design effect from all angles, or when there is a sudden change in lighting conditions, the marker exceeds the image acquisition field of view, or the image becomes blurry, situations may arise where the operating component obstructs the preset marker or the preset marker faces away from the image acquisition device. This can lead to the failure of target positioning feature information recognition, making it impossible to directly calculate the pose data of the target packaging carrier. At this time, since the operating component is still grasping the target packaging carrier, the relative position of the two has not changed. The server can obtain the most recently recorded relative spatial transformation relationship before the marker is lost and combine it with the feature point position information tracked in the current frame to perform pose inverse compensation. Here, pose inverse compensation refers to using the known spatial coordinates of the operating component as the starting point of the new reference frame, and by inversely applying the previously established relative spatial transformation relationship, deriving and calculating the current spatial position and orientation of the target packaging carrier, thereby obtaining the estimated pose data of the target packaging carrier in real three-dimensional space.
[0069] Thus, this embodiment can ensure the spatial alignment continuity between the virtual 3D model and the target packaging carrier in the mixed reality rendering engine even when the preset mark is temporarily invisible. For example, when a user flips the packaging box, causing the preset mark to be obscured by their finger, the server cannot directly recognize the preset mark. At this time, the estimated pose of the packaging box can be calculated in reverse compensation based on the current position of the finger and the previously established relative spatial transformation relationship, thus avoiding interruption or jitter in the mixed reality preview.
[0070] This embodiment introduces feature point extraction of operating components and establishes the relative spatial transformation relationship between virtual and real objects. It can use the position of the operating components to perform pose inverse deduction compensation, which solves the problem of intermittent loss of preset marks caused by factors such as hand occlusion and viewpoint changes in mixed reality application scenarios. Thus, this embodiment can improve the robustness and continuity of pose tracking, thereby improving the stability and visual coherence of 3D virtual-real mapping in actual interactive scenarios. It allows designers to freely flip the packaging entity without deliberately avoiding occlusion actions, optimizing the human-computer interaction experience and intelligence level of the intelligent packaging design system.
[0071] In step S502, after acquiring the pose data, the server can invoke the mixed reality rendering engine to spatially align the target packaging carrier and its corresponding target virtual 3D model based on the pose data. Specifically, the server places the target virtual 3D model in the digital coordinate system of the mixed reality rendering engine and adjusts the pose of the target virtual 3D model using the calculated pose data to make it consistent with the position and angle of the target packaging carrier in the real environment.
[0072] After aligning the virtual space with the real space, the server will update the packaging design image as a texture map and map it onto the surface of the spatially aligned target virtual 3D model to generate the corresponding 3D preview result.
[0073] This embodiment enhances the 3D spatial verification capability of intelligent packaging design solutions before implementation by utilizing pose tracking technology and a mixed reality rendering engine. It addresses the limitations of traditional AIGC packaging design tools, which only allow offline previews on 2D screens and lack real-world environmental perception and spatial interaction. This enables designers to intuitively verify the spatial proportions, perspective effects, and visual harmony of packaging designs within a real-world environment during the design phase. Consequently, this embodiment avoids designers discovering design flaws only in the later prototyping stage, saving on trial-and-error material costs and time, and improving the iterative quality and production verification efficiency of intelligent packaging design.
[0074] In step S105, since packaging design typically requires multiple rounds of refinement, this embodiment can consider the currently generated updated packaging design image as the starting point for the next round of optimization (i.e., a new initial packaging design image). Users can re-trigger the AI visual analysis and evaluation in step S102 based on visual deviations discovered in the mixed reality 3D preview or new inspiration, or directly perform secondary image editing in step S103. This iterative cycle involving evaluation, modification, and 3D verification will continue until the design fully meets expectations. When the server receives the final confirmation instruction from the client, the iteration ends, and the target packaging design image is determined based on the updated packaging design image generated in the last iteration.
[0075] In some embodiments, please refer to Figure 7 The method provided in this application also includes: Step S701: Obtain the packaging design sketch data, prompt text data, optimization feedback data, and corresponding updated packaging design image obtained from each round of iteration. Step S702: Establish a mapping relationship between packaging design sketch data, prompt word data, optimization feedback data and corresponding updated packaging design images, and identify and bind the associated packaging design sketch data, prompt word data, optimization feedback data and updated packaging design images according to the mapping relationship, and generate historical packaging design version data and corresponding version thumbnails; Step S703: Send the version thumbnail to the client so that the client can display the version thumbnail.
[0076] In step S701, the actual packaging design needs to go through multiple rounds of iteration and fine-tuning. After the user completes a design modification action, the server can automatically capture the packaging design sketch data, prompt text data, optimization feedback data and update the packaging design image for the current round.
[0077] In step S702, after acquiring the aforementioned data, the server can establish a mapping relationship between packaging design sketch data, cue word data, optimization feedback data, and corresponding updated packaging design images in the underlying database. Based on this mapping relationship, the server identifies and binds the associated packaging design sketch data, cue word data, optimization feedback data, and updated packaging design images. Specifically, the server can assign a unique global identifier to packaging design sketch data, cue word data, optimization feedback data, and updated packaging design images belonging to the same iteration. For example, when a user adds the cue word "splashing juice" and its corresponding sketch in version V2, the server can bind these input parameters to the final image of V2 and the AI's evaluation of that image, packaging and storing them as traceable historical packaging design version data. Simultaneously, to facilitate visual recognition by client users, the server can compress and sample the main visual image of each modified version, automatically generating a lightweight version thumbnail.
[0078] In step S703, the server sends the version thumbnail to the client so that the client can display the version thumbnail in the packaging design interface, such as... Figure 4As shown, version thumbnails can be displayed within the historical version management area (450 pixels) in the packaging design interface. This allows users to intuitively browse the entire design evolution without leaving the current main view. More importantly, because the version thumbnails are intrinsically linked to historical packaging design version data, when a user clicks or selects a version thumbnail on the client, the server can retrieve and restore the corresponding original sketches, prompts, and AI feedback. This facilitates horizontal comparison reviews, one-click rollback, or secondary creation and reuse based on specific historical version branches.
[0079] This embodiment transforms the fragmented design process into a structured and traceable digital asset through underlying mapping binding and front-end thumbnail visualization. Compared to the traditional design workflow, where parameters and review comments from previous iterations are easily lost, leading to waste of design assets and difficulty in retrospective modifications, this embodiment allows for the reuse and rollback of historical versions at any time, reducing communication and trial-and-error costs and improving the efficiency of multi-round iterations in packaging design and the scientific nature of version management.
[0080] In some embodiments, please refer to Figure 8 , Figure 8 This is an architecture diagram of the packaging design system provided in this application, such as... Figure 8 As shown, the packaging design system consists of a front-end application, a back-end, and a mixed reality preview and mapping module. First, the front-end receives user voice and text prompts and sketch data through a multimodal interaction layer and a 2D design editing layer, and sends this data to the back-end via a data request interface. Simultaneously, the version and asset management module stores historical versions. The back-end uses an image generation service to receive sketches, masks, prompts, and editing hints, and generates packaging design images based on this data, returning the image results. Simultaneously, the AI dialogue and visual analysis module performs visual understanding of the current design and outputs actionable design optimization suggestions. The AI task scheduling module is responsible for overall request routing and state synchronization. Finally, the mixed reality preview and mapping module uses a camera to recognize preset markers in real time to estimate the physical packaging pose, outputting the packaging's spatial position and orientation. The current design is mapped as a texture onto a virtual 3D model in real time, and the preset markers align the 3D virtual model with the real object, outputting a mixed reality preview result. This preview result is returned to the front-end via an interactive feedback interface, allowing users to check the design effect in the mixed reality view and make further modifications.
[0081] In some embodiments, please refer to Figure 9 , Figure 9 This is another flowchart of the packaging design method provided in this application, such as Figure 9As shown, in the creative conception stage, users communicate with a real-time conversational AI based on a large language model through a multimodal interactive interface to clarify the initial style and visual direction. Next, in the sketch and prompt input stage, the client obtains the user's sketch outlines and text descriptions on the packaging design interface. Then, in the image generation stage, the client packages the multimodal data and sends it to the backend image generation model to infer and generate the initial packaging design image. Simultaneously, the backend can automatically store the current design image and associated UV mapping data, forming a traceable historical version. Finally, in the AI feedback stage, the AI processes the generated image... Visual features are extracted and analyzed, and real-time modification suggestions are made in terms of color and layout based on preset design rules. Users edit and redraw images based on AI feedback data. Then, in the mixed reality mapping preview stage, preset marks on the physical packaging are identified in real time through image acquisition devices and their spatial poses are estimated. The latest design is then rendered as a texture onto the virtual 3D model in real time to achieve virtual-real mapping. Finally, after observing and comparing the output, it is determined whether the user is satisfied with the current design result. If not, the process returns to the editing and revision stage along the dotted line. If the user confirms satisfaction, the final packaging design scheme data is confirmed.
[0082] Steps S101 to S105 of this embodiment generate an initial packaging design image based on packaging design sketch data and prompt word data by calling an image generation model, reducing the initial design threshold and trial-and-error costs. Subsequently, a large language model is called to evaluate the initial image and output optimization feedback data, overcoming the shortcomings of existing technologies such as the lack of a visual evaluation loop and reliance on blind manual testing. Furthermore, by receiving image editing and optimization instructions from the client and performing image update operations to obtain an updated packaging design image, collaborative modification with human intervention and AI assistance is achieved, improving the accuracy and efficiency of design modifications. Next, a mixed reality rendering engine is called, combined with the target virtual 3D model, to perform a 3D preview operation, solving the problem of traditional design schemes lacking spatial verification capabilities and difficulty in intuitively examining the actual mapping effect. Finally, the updated packaging design image undergoes at least one round of iterative processing based on evaluation, update, and 3D preview until a final confirmation instruction is received. Thus, this application constructs an integrated intelligent packaging design workflow, improving the continuity of the entire packaging design chain, thereby enhancing the intelligence level of the packaging design process.
[0083] Please see Figure 10 This application also provides a packaging design device 1000, which can implement the above-described packaging design method. The packaging design device 1000 includes: The first generation unit 1010 is used to acquire the packaging design sketch data sent by the client and the prompt words data used to render and describe the packaging design sketch data, and call the preset image generation model to generate an initial packaging design image based on the packaging design sketch data and the prompt words data. The sending unit 1020 is used to call a preset large language model to evaluate the initial packaging design image, generate corresponding optimization feedback data based on the evaluation results, and send the optimization feedback data to the client. The update unit 1030 is used to receive the image editing optimization instruction sent by the client based on the optimization feedback data, and to perform an image update operation on the initial packaging design image according to the image editing optimization instruction to obtain the updated packaging design image; The second generation unit 1040 is used to acquire the carrier image data corresponding to the target packaging carrier, and call the preset mixed reality rendering engine to perform a three-dimensional preview operation based on the updated packaging design image, carrier image data and the target virtual three-dimensional model corresponding to the target packaging carrier, and generate the corresponding three-dimensional preview result. The determining unit 1050 is used to take the updated packaging design image as a new initial packaging design image, and perform at least one round of iterative processing based on at least one of the evaluation operation, image update operation and 3D preview operation until the final confirmation instruction sent by the client is received, and determine the target packaging design image based on the final updated packaging design image.
[0084] The specific implementation method of this packaging design device is basically the same as the specific implementation method of the above-mentioned packaging design method, and will not be described again here.
[0085] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described packaging design method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.
[0086] Please see Figure 11 , Figure 11 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 1101 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 1102 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1102 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1102 and is called and executed by the processor 1101 using the packaging design method of the embodiments of this application. Input / output interface 1103 is used to implement information input and output; The communication interface 1104 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 1105 transmits information between various components of the device (e.g., processor 1101, memory 1102, input / output interface 1103, and communication interface 1104); The processor 1101, memory 1102, input / output interface 1103 and communication interface 1104 are connected to each other within the device via bus 1105.
[0087] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described packaging design method.
[0088] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.
[0089] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.
[0090] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.
[0091] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.
[0092] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.
[0093] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0094] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.
[0095] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.
[0096] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0097] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0098] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0099] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.
Claims
1. A packaging design method, characterized in that, Applied to the server side, the method includes: The system acquires packaging design sketch data sent by the client and prompt words used to render and describe the packaging design sketch data, and calls a preset image generation model to generate an initial packaging design image based on the packaging design sketch data and the prompt words data. The system calls a preset large language model to evaluate the initial packaging design image, generates corresponding optimization feedback data based on the evaluation results, and sends the optimization feedback data to the client. The system receives an image editing and optimization instruction from the client, which is based on the optimization feedback data, and performs an image update operation on the initial packaging design image according to the image editing and optimization instruction to obtain an updated packaging design image. The carrier image data corresponding to the target packaging carrier is obtained, and a preset mixed reality rendering engine is called to perform a three-dimensional preview operation based on the updated packaging design image, the carrier image data, and the target virtual three-dimensional model corresponding to the target packaging carrier, and generate the corresponding three-dimensional preview result. The updated packaging design image is used as the new initial packaging design image. At least one round of iterative processing is performed based on at least one of the evaluation operation, the image update operation, and the 3D preview operation until the final confirmation instruction sent by the client is received. The target packaging design image is then determined based on the final updated packaging design image.
2. The method according to claim 1, characterized in that, The invocation of a preset mixed reality rendering engine performs a 3D preview operation based on the updated packaging design image, the carrier image data, and the target virtual 3D model corresponding to the target packaging carrier, generating a corresponding 3D preview result, including: The target positioning feature information corresponding to the preset mark on the surface of the target packaging carrier is identified in the carrier image data, and the pose data of the target packaging carrier in real three-dimensional space is calculated based on the target positioning feature information. The preset mixed reality rendering engine is invoked to spatially align the target packaging carrier and the corresponding target virtual 3D model based on the pose data, and the updated packaging design image is mapped as a texture map onto the surface of the spatially aligned target virtual 3D model to generate the corresponding 3D preview result.
3. The method according to claim 2, characterized in that, The carrier image data also includes image data containing both the target packaging carrier and the operating components for operating the target packaging carrier. The step of identifying target positioning feature information corresponding to preset marks on the surface of the target packaging carrier in the carrier image data, and calculating the pose data of the target packaging carrier in real three-dimensional space based on the target positioning feature information, includes: Extract the feature point location information of the operating component from the carrier image data; When the target positioning feature information corresponding to the preset mark on the surface of the target packaging carrier is successfully identified in the carrier image data, the pose data of the target packaging carrier in the real three-dimensional space is calculated based on the target positioning feature information, and the relative spatial transformation relationship between the feature point position information and the calculated pose data is determined. When the recognition of the target positioning feature information fails, pose inverse compensation is performed based on the relative spatial transformation relationship and the feature point position information to obtain the pose data of the target packaging carrier in real three-dimensional space.
4. The method according to claim 1, characterized in that, The process of obtaining the packaging design sketch data sent by the client includes: Receive natural language instructions sent by the client, and perform semantic analysis on the natural language instructions to obtain the packaging design intent; Based on the packaging design intent, generate packaging design inspiration suggestions and obtain packaging design sketch data obtained by the client based on the packaging design inspiration suggestions.
5. The method according to claim 1, characterized in that, The step of calling a preset image generation model to generate an initial packaging design image based on the packaging design sketch data and the prompt word data includes: Obtain background image data corresponding to the display background of the target packaging carrier, and extract features from the background image data to obtain the corresponding three-dimensional background environment features; The three-dimensional background environment features are mapped to corresponding environmental constraint semantic labels, and fused prompt word data is generated based on the environmental constraint semantic labels and the prompt word data; The preset image generation model is invoked to generate an initial packaging design image based on the packaging design sketch data and the fused prompt word data.
6. The method according to claim 1, characterized in that, The method further includes: Acquire the packaging design sketch data, prompt text data, optimization feedback data, and corresponding updated packaging design images obtained from each round of iteration; Establish a mapping relationship between the packaging design sketch data, the prompt word data, the optimization feedback data, and the corresponding updated packaging design image, and identify and bind the associated packaging design sketch data, prompt word data, optimization feedback data, and updated packaging design image according to the mapping relationship to generate historical packaging design version data and corresponding version thumbnails; The version thumbnail is sent to the client so that the client can display the version thumbnail.
7. The method according to claim 1, characterized in that, The process of displaying the optimization feedback data on the client includes the following steps: Displays a packaging design interface, which includes an evaluation feedback area; The optimized feedback data is displayed in the evaluation feedback area, and the optimized feedback data is read aloud via voice.
8. The method according to claim 1, characterized in that, The process of invoking a preset large language model to evaluate the initial packaging design image and generating corresponding optimization feedback data based on the evaluation results includes: A preset visual saliency prediction model is invoked to generate a corresponding visual attention heatmap based on the initial packaging design image; Target design elements are identified in the initial packaging design image, and the attentional overlap of the target design elements in the visual attention heatmap is calculated. When the attention overlap is lower than a preset threshold, the initial packaging design image is evaluated by calling a preset large language model, using the visual attention heatmap and the spatial distribution information of the target design element as constraints, and corresponding optimization feedback data is generated based on the evaluation results.
9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the packaging design method according to any one of claims 1 to 8.
10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the packaging design method according to any one of claims 1 to 8.
