An automated method of matching picture templates and assets

Abstract

The present application relates to the technical field of computer vision, in particular to a kind of automatic matching method of picture template and material, comprising the following steps: S1: matching, in the container in template unit, select a suitable material picture for each container;S2: positioning and cutting, the matched picture is cut to the size suitable for filling template.The present application analyzes the container demand through language big model, score mechanism dynamically balances picture type, main body, interaction, adapts to different template demand, and uses multi-modal model to detect material picture main body, expands boundary to maximize to retain background, according to template size proportion, central cutting, avoids deformation, ensures visual aesthetic appearance.At the same time, from matching to cutting whole process does not need manual intervention, supports API access e-commerce system, efficiency is improved by more than 90%, adapts to the template rule of multiple overseas e-commerce platform, reduces cross-platform operation cost.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to an automated matching method for image templates and materials. Background Technology

[0002] In the process of automatically generating product detail page images, to improve the efficiency of the automation process, for different image templates created based on product type and sales platform, it is necessary to automatically match one or more suitable material images for each template to generate the images required for the product detail page. Each area in a template that holds material images is called an image container, and the material content and size required for different containers in different templates are not exactly the same. At the same time, in order to ensure the aesthetics of the generated images, in addition to automatically selecting suitable images from the given material pool, it is also necessary to crop them to the size corresponding to the container and keep the main subject in the image in the center to avoid image compression or stretching distortion, or the subject being offset in the image.

[0003] In existing technologies, some solutions rely on manual selection and cropping of source images to fit templates, which is not only inefficient but also fails to meet the batch processing needs of multiple platforms and templates, resulting in high cross-platform operation costs. Other automated solutions select images solely through simple keyword matching or size comparison, failing to accurately identify the container's deeper requirements for image type, main object, and object interaction relationships, leading to low matching accuracy and poor adaptation of generated images to templates. In the image cropping stage, traditional methods often use fixed-ratio scaling or simple cropping, lacking precise positioning of the image subject and failing to intelligently handle background features, frequently resulting in subject misalignment, background redundancy, or missing key information, affecting the visual presentation of product detail pages. Furthermore, existing solutions lack effective result verification mechanisms, easily leading to container matching omissions and difficulty adapting to the different template rules of various overseas e-commerce platforms, further limiting the practicality and applicability of automated processes. Summary of the Invention

[0004] The purpose of this invention is to provide an automated matching method for image templates and materials to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: an automated matching method for image templates and materials, comprising the following steps: S1: Matching, selects a suitable image for each container in the template; S2: Positioning and cropping, cropping the matched image to fit the size of the template.

[0006] Preferably, step S1 includes simultaneously inputting the input container list and the material image information list into the language big model, and the language big model analyzes the image requirements of each container, as well as the image type of the material image, the main object in the image, and the interaction relationship between different objects in the image.

[0007] Preferably, step S1 further includes the language model scoring the matching degree of each material image based on the image matching requirements of the container, including image type, main object, and object interaction relationship, and calculating the total matching score according to the ratio of image type × 40% + main object × 40% + interaction relationship × 20%.

[0008] Preferably, step S1 further includes the language big model selecting the material image with the highest total score for each container and outputting the container ID and the corresponding image ID.

[0009] Preferably, step S1 further includes checking the output of the language model; if a container lacks a matching result, a retry is performed to ensure that each container matches the corresponding source image.

[0010] Preferably, step S2 includes performing target detection on the source image matched to each container using a target detection model to obtain target detection results.

[0011] Preferably, step S2 further includes determining whether the target detection result is empty; if no target is detected, the original image boundary is used as the bbox; if a target is detected, the bbox with the largest area is selected.

[0012] Preferably, step S2 further includes expanding the boundary of the largest bbox, specifically: Using this bounding box as a reference, convert the image to grayscale and expand outwards every three pixels along each side of the bounding box. During the expansion process, the average pixel value of the current pixel row and column is calculated, as well as the proportion of similar pixels that differ from the average pixel value by less than 8 and the proportion of white pixels with a pixel value greater than 250.

[0013] Preferably, the stopping condition for the boundary expansion includes: Condition 1: When the proportion of similar pixels is greater than 0.93 or the proportion of white pixels is greater than 0.8, calculate the average difference between the current pixel row and column and the rows and columns three pixels apart. If the difference is greater than 10, stop expanding. Condition 2: If condition 1 is not met, calculate the mean variance of each color channel in rows and columns that are three pixels apart in the original color image. If the mean is less than 10, stop the expansion.

[0014] Preferably, step S2 further includes selecting the required size ratio from the template container size lookup table according to the given template style tag, cropping the image with the center point of the original bbox as the center and the expanded bbox as the boundary to obtain an image of the target size ratio, and directly filling the container corresponding to the template into the cropped image for subsequent image production.

[0015] Compared with the prior art, the beneficial effects of the present invention are: This invention uses a large language model to analyze container requirements, and a scoring mechanism dynamically balances image type, subject, and interaction relationships to adapt to different template requirements. It also utilizes a multimodal model to detect the subject of the source image, expands boundaries to maximize background preservation, and centers the image according to the template size proportions to avoid distortion and ensure visual appeal. Furthermore, the entire process from matching to cropping requires no manual intervention, supports API integration with e-commerce systems, improves efficiency by over 90%, and adapts to template rules of multiple overseas e-commerce platforms, reducing cross-platform operation costs. Detailed Implementation

[0016] The technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] This invention provides a technical solution: an automated matching method for image templates and materials, comprising the following steps: S1: Matching, selects a suitable image for each container in the template; S2: Positioning and cropping, cropping the matched image to fit the size of the template.

[0018] S1 specifically includes the following four steps: S1-1: Input the list of containers and the list of source images into the Gemini 2.0 Flash language model simultaneously, and let it analyze the image requirements of each container in the container list and the image information in the source image information list, including the image type, the main object in the image, and the interaction relationship between different objects in the image.

[0019] Gemini 2.0 Flash, officially released by Google in February 2025, is a next-generation AI language model and a core member of the Gemini 2.0 series. This model utilizes the Transformer architecture and is built on Google's custom sixth-generation TPU Trillium hardware, achieving significant performance improvements while maintaining low latency. Key technical features include support for ultra-long context windows of up to 1 million tokens, enabling the processing of large-scale data and maintaining the coherence of long documents. The model possesses native multimodal capabilities, simultaneously handling multiple input formats such as text, images, audio, and video, and supports multimodal output, including text, images, and native text-to-speech (TTS) audio generation. In key benchmark tests, Gemini 2.0 Flash's response speed is twice that of its predecessor, Gemini 1.5 Pro, demonstrating excellent performance in tasks such as multimodal reasoning, mathematical reasoning, and code generation. The model also features native tool invocation capabilities, seamlessly integrating external tools such as Google Search and code execution, enhancing information retrieval and task execution capabilities. Application Scenarios: This model is designed for large-scale, high-frequency tasks and is suitable for various fields such as real-time dialogue, intelligent assistants, code generation, and content creation. Developers can access the model through Google AI Studio and the Vertex AI platform.

[0020] S1-2: The Gemini 2.0 Flash language model scores each image based on its matching requirements for each container in the container list, considering the image type, the main object in the image, and the degree of matching between different objects in the image. The total score is calculated according to the following ratio: image type × 40% + main object in the image × 40% + interaction between different objects in the image × 20%.

[0021] S1-3. Have the language big model Gemini 2.0 Flash select the highest-scoring image for each container and output the container ID and its corresponding image ID.

[0022] S1-4. Check the output of the language large model Gemini 2.0 Flash. If a container is found to be missing a matching result, retry to ensure that each container can find a matching image.

[0023] Step S2 specifically includes the following steps: S2-1: Perform object detection on the image matched for each container, and obtain the object detection results through the object detection model GroundingDINO.

[0024] Grounding DINO, developed by IDEA Research, is an advanced zero-shot object detection model that combines the Transformer-based detector DINO with Grounded Pre-Training to enable the detection of arbitrary objects based on human input (such as category names or identifiers). The model employs a dual-encoder-single-decoder architecture, including an image backbone network (Swin Transformer) and a text backbone network (BERT). It achieves deep visual and linguistic modality fusion through feature enhancers, a language-guided query selection module, and a cross-modal decoder. This model supports end-to-end optimization, requires no manual module design, and can be widely applied to scenarios requiring flexible object detection, such as autonomous driving, robot vision, and image annotation.

[0025] S2-2: Determine whether the target detection result is empty. If no target is detected, directly select the image boundary of the original image as the bounding box and proceed to S2-4. If a target is detected, select the bounding box with the largest area and proceed to S2-3.

[0026] S2-3: Expand the bounds of the bounding box (bbox) to maximize the use of the image material and obtain a more aesthetically pleasing image. Boundary expansion uses the bbox obtained in S2-2 as a base. After converting the image to grayscale, each edge of the bbox is expanded. The specific expansion rules are as follows: Using the bounding box as a reference, first convert the image to grayscale, then expand each side of the bounding box outward at intervals of three pixels. During the expansion process, the average pixel value of the current pixel row and column needs to be calculated, and then the proportion of similar pixels with a difference of less than 8 from the average pixel value and the proportion of white pixels with a pixel value greater than 250 are obtained respectively. The stopping condition for boundary expansion is as follows: If the proportion of similar pixels is greater than 0.93 or the proportion of white pixels is greater than 0.8, the average difference between the current pixel row / column and the pixel rows / columns three pixels away is calculated, and expansion stops when the difference is greater than 10; if the proportion of similar pixels is not greater than 0.93 and the proportion of white pixels is not greater than 0.8, the mean variance of each color channel in the pixel rows / columns three pixels away from the current pixel row / column in the original color image is calculated, and expansion stops when the mean is less than 10.

[0027] S2-4: Based on the given template style tags, select the required size ratio from the known template container size lookup table, and crop the image using the center point of the original bounding box as the center and the expanded bounding box as the boundary to obtain an image of the target size ratio. This ensures that the object is centered in the image and that the image will not be distorted after being filled into the template. Finally, you will get a container and the cropped image. You can directly put the image into the container of the image template to proceed with the subsequent image production process.

[0028] In this scheme, the bounding box (bbox) is a key concept in object detection, used to represent the position and size of a target object in an image. The bounding box is typically a rectangle, and its four parameters define its position in the image. There are generally two formats: the first is xyxy, representing the coordinates of the top-left corner (x1, y1) and the bottom-right corner (x2, y2); the second is xywh, representing the coordinates of the top-left corner (x, y), width, and height.

[0029] This invention uses a large language model to analyze container requirements, and a scoring mechanism dynamically balances image type, subject, and interaction relationships to adapt to different template requirements. It also utilizes a multimodal model to detect the subject of the source image, expands boundaries to maximize background preservation, and centers the image according to the template size proportions to avoid distortion and ensure visual appeal. Furthermore, the entire process from matching to cropping requires no manual intervention, supports API integration with e-commerce systems, improves efficiency by over 90%, and adapts to template rules of multiple overseas e-commerce platforms, reducing cross-platform operation costs.

[0030] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An automated matching method for image templates and materials, characterized in that, It includes the following steps: S1: Matching, selects a suitable image for each container in the template; S2: Positioning and cropping, cropping the matched image to fit the size of the template.

2. The automated matching method for image templates and materials according to claim 1, characterized in that: S1 includes simultaneously inputting the list of input containers and the list of material images into the language model, and the language model analyzes the image requirements of each container, as well as the image type of the material images, the main objects in the images, and the interaction relationships between different objects in the images.

3. The automated matching method for image templates and materials according to claim 1, characterized in that: S1 further includes the language model scoring the matching degree of each material image based on the image matching requirements of the container, including image type, main object, and object interaction relationship, and calculating the total matching score according to the ratio of image type × 40% + main object × 40% + interaction relationship × 20%.

4. The automated matching method for image templates and materials according to claim 1, characterized in that: S1 also includes the language model selecting the image with the highest total score for each container and outputting the container ID and the corresponding image ID.

5. The automated matching method for image templates and materials according to claim 1, characterized in that: S1 further includes checking the output of the language model. If a container is missing a matching result, a retry is performed to ensure that each container matches the corresponding image.

6. The automated matching method for image templates and materials according to claim 1, characterized in that: S2 includes performing target detection on the material image matched for each container using a target detection model to obtain the target detection result.

7. The automated matching method for image templates and materials according to claim 1, characterized in that: The S2 further includes determining whether the target detection result is empty. If no target is detected, the original image boundary is used as the bbox. If a target is detected, the bbox with the largest area is selected.

8. The automated matching method for image templates and materials according to claim 1, characterized in that: S2 also includes expanding the boundaries of the largest bbox, specifically: Using this bounding box as a reference, convert the image to grayscale and expand outwards every three pixels along each side of the bounding box. During the expansion process, the average pixel value of the current pixel row and column is calculated, as well as the proportion of similar pixels that differ from the average pixel value by less than 8 and the proportion of white pixels with a pixel value greater than 250.

9. The automated matching method for image templates and materials according to claim 8, characterized in that: The stopping conditions for the boundary expansion include: Condition 1: When the proportion of similar pixels is greater than 0.93 or the proportion of white pixels is greater than 0.8, calculate the average difference between the current pixel row and column and the rows and columns three pixels apart. If the difference is greater than 10, stop expanding. Condition 2: If condition 1 is not met, calculate the mean variance of each color channel in rows and columns that are three pixels apart in the original color image. If the mean is less than 10, stop the expansion.

10. The automated matching method for image templates and materials according to claim 1, characterized in that: S2 further includes selecting the required size ratio from the template container size lookup table according to the given template style tag, cropping the image with the center point of the original bbox as the center and the expanded bbox as the boundary to obtain an image of the target size ratio, and directly filling the container corresponding to the template into the cropped image for subsequent image production.