Image processing method and device, storage medium and electronic device

Abstract

The application discloses an image processing method and device, a storage medium and an electronic device. The method comprises the following steps: in response to an image processing task received, obtaining a task type identifier and an image resource identifier; reading image metadata of an original image based on the image resource identifier; preprocessing the original image to obtain an intermediate image of a target size; extracting spatial distribution information of each object in the intermediate image based on the task type identifier to form anchor point position features; dynamically selecting a target text template from a preset template library based on the image metadata, the task type identifier and the anchor point position features; inputting the target text template and the intermediate image into a target processing model to execute image processing logic corresponding to the task type identifier, and obtaining a target processing result. The application solves the technical problem of poor quality in the image processing process.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more specifically, to an image processing method and apparatus, a storage medium and an electronic device. Background Technology

[0002] Current AIGC (Artificial Intelligence Generated Content) image editing technologies primarily rely on object detection bounding boxes or full-image semantic segmentation, which is insufficient to meet the pixel-level precision requirements for interactive processing based on any point in the image. In non-standard scale images, the model is prone to visual distortion and "phantom readings," leading to significant deviations in visual anchor point localization.

[0003] Meanwhile, limitations in resolution and poor prompt design (such as high coupling due to hard coding, lack of version control and dynamic adaptation capabilities) can easily lead to geometric misalignment and semantic illusions, making it difficult to guarantee the quality and reproducibility of image generation. Furthermore, large-scale material production requires parallelizable and traceable generation processes that can automatically select appropriate prompts for different scenarios, which can easily trigger semantic illusions, resulting in poor image quality during image processing.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This application provides an image processing method and apparatus, a storage medium and an electronic device to at least solve the technical problem of poor image quality that occurs during image processing.

[0006] According to one aspect of the embodiments of this application, an image processing method is provided, comprising: in response to a received image processing task, acquiring a task type identifier and an image resource identifier; reading image metadata of an original image based on the image resource identifier; preprocessing the original image to obtain an intermediate image of a target size; extracting spatial distribution information of each object in the intermediate image based on the task type identifier to form anchor point position features; dynamically selecting a target text template from a preset template library based on the image metadata, the task type identifier, and the anchor point position features; and inputting the target text template and the intermediate image into a target processing model to execute image processing logic corresponding to the task type identifier and obtain a target processing result.

[0007] According to another aspect of the embodiments of this application, an image processing apparatus is also provided, comprising: a first acquisition unit, configured to acquire a task type identifier and an image resource identifier in response to a received image processing task; a reading unit, configured to read image metadata of an original image based on the image resource identifier; a preprocessing unit, configured to preprocess the original image to obtain an intermediate image of a target size; and extract spatial distribution information of each object in the intermediate image based on the task type identifier to form anchor point position features; a filtering unit, configured to dynamically filter a target text template from a preset template library based on the image metadata, the task type identifier, and the anchor point position features; and a first processing unit, configured to input the target text template and the intermediate image into a target processing model to execute image processing logic corresponding to the task type identifier and obtain a target processing result.

[0008] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program for executing the above-described image processing method when the electronic device is run.

[0009] According to another aspect of the embodiments of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the steps of the above-described method.

[0010] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to execute the image processing method described above through the computer program.

[0011] Using the embodiments provided in this application, by reading image metadata and preprocessing the original image, the non-standard aspect ratio original image is transformed into an intermediate image of the target size for the adaptive model, avoiding visual distortion caused by abnormal aspect ratio. Based on the task type identifier, the spatial distribution information of the intermediate image is extracted to form anchor point position features, achieving precise mapping from user interaction coordinates to pixel-level positions. Combining image metadata, task type, and anchor point position features, target text templates are dynamically selected from a preset template library, ensuring a high semantic fit between the Prompt and the current image processing task and anchor point position, suppressing semantic illusions caused by instruction mismatch. This allows the target processing model to accurately execute image processing logic matching different processing tasks under the prompt of the target text template. In other words, by using dynamically adapted constraints to guide model generation, while ensuring positioning accuracy, the structure, semantic accuracy, and geometric consistency of the generated results are improved, solving the technical problems of large positioning deviation, uncontrollable generation, and rigid processes in traditional solutions, thus achieving the technical effect of improving image processing quality. Attached Figure Description

[0012] The accompanying drawings, which are provided to further illustrate this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application.

[0013] Figure 1 This is a schematic diagram illustrating an application scenario of an optional image processing method according to an embodiment of this application;

[0014] Figure 2 This is a flowchart of an optional image processing method according to an embodiment of this application;

[0015] Figure 3 This is an overall architecture diagram of an optional image processing method according to an embodiment of this application;

[0016] Figure 4 This is an overall flowchart of an optional image processing method according to an embodiment of this application;

[0017] Figure 5 It is the overall processing flow for image processing tasks when locating visual anchor points;

[0018] Figure 6 This refers to the overall processing flow when an image processing task generates image layers;

[0019] Figure 7 This is the overall processing flow of the Prompt dynamic template decision model;

[0020] Figure 8 This is a schematic diagram of the structure of an optional image processing apparatus according to an embodiment of this application;

[0021] Figure 9 This is a schematic diagram of the structure of an optional electronic device according to an embodiment of this application. Detailed Implementation

[0022] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0023] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0024] The technical solutions in this application will comply with legal regulations during implementation. When operating according to the technical solutions in the embodiments, the data used will not involve user privacy, ensuring that the operation process is compliant and legal while guaranteeing data security. In addition, when the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use, and processing of related data must comply with the relevant regulations and standards of the relevant countries or regions.

[0025] According to one aspect of the embodiments of this application, an image processing method is provided. As an optional implementation, the above-described image processing method may be applied to, but is not limited to, [examples of other methods]. Figure 1 The application scenarios shown are as follows. In, for example... Figure 1 In the application scenario shown, the target terminal 102 can communicate with the server 106 via network 104, but is not limited to this. The server 106 can perform operations on the database 108, such as write or read data operations. The target terminal 102 may include, but is not limited to, a human-computer interaction screen, a processor, and a memory. The human-computer interaction screen may be used to display images of various main objects and background images in the original image obtained after processing using the technical solution of this application on the target terminal 102. The processor may be used to respond to the human-computer interaction operation, execute corresponding operations, or generate corresponding instructions and send the generated instructions to the server 106. The memory is used to store relevant processing data, such as task type identifiers, image resource identifiers, and anchor point position features.

[0026] Optionally, in this embodiment, the target terminal can be a terminal configured with a target client, which may include, but is not limited to, at least one of the following: mobile phone (such as Android phone, iOS phone, etc.), laptop computer, tablet computer, PDA, MID (Mobile Internet Devices), PAD, desktop computer, smart TV, etc. The target client may be a video client, instant messaging client, browser client, educational client, etc. The network may include, but is not limited to, wired network and wireless network, wherein the wired network includes: local area network, metropolitan area network and wide area network, and the wireless network includes: Bluetooth, WIFI and other networks that enable wireless communication. The server may be a single server, a server cluster composed of multiple servers, or a cloud server.

[0027] The technical solution of this application is applicable to intelligent image editing and generation scenarios, such as multi-platform adaptation of product images and multi-scenario content production for social media.

[0028] To address this problem, an image processing method is proposed in this application embodiment. Figure 2 This is a flowchart of an image processing method according to an embodiment of the present application, which includes the following steps S202 to S208.

[0029] It should be noted that the image processing methods shown in steps S202 to S208 can be, but are not limited to, executed by an electronic device, wherein the electronic device can be, but is not limited to, as shown in the example below. Figure 1 The target terminal or server shown.

[0030] Step S202: In response to the received image processing task, obtain the task type identifier and image resource identifier;

[0031] Step S204: Based on the image resource identifier, read the image metadata of the original image;

[0032] Step S206: Preprocess the original image to obtain an intermediate image of the target size; and extract the spatial distribution information of each object in the intermediate image based on the task type identifier to form anchor point position features;

[0033] Step S208: Based on image metadata, task type identifier, and anchor point location features, dynamically select target text templates from the preset template library;

[0034] Step S210: Input the target text template and intermediate image into the target processing model to execute the image processing logic corresponding to the task type identifier and obtain the target processing result.

[0035] The image metadata includes, but is not limited to, the original dimensions of the original image, primarily the original width W and height H.

[0036] To facilitate understanding, let's first combine... Figure 3 The overall architecture diagram shown provides a brief overview of the implementation process of the above image processing method.

[0037] like Figure 3 As shown, the overall architecture for the above image processing method can, but is not limited to, adopt a modular design, including a front-end presentation layer, an application service layer, and a data persistence layer.

[0038] The front-end presentation layer is used to transform the user's ambiguous interactions into structured, machine-executable instructions, that is, to collect the user's interaction entry point (such as the coordinates of the click location) and the image processing intent. For example, on an image preview, the user clicks a specified visual anchor point, and the front-end captures the user's click coordinates (x, y); the user selects the operation intent, such as identifying the anchor point object or generating / cutting out layers; in addition, the user can also select or input specific prompt constraints, target size, or template preferences, etc.

[0039] In addition, the front-end presentation layer is responsible for generating and carrying session context information, including seqID (session ID), userCode (user identifier), and assetID (image resource ID), ensuring the traceability of subsequent operations. It can also receive structured recognition results returned from the backend (such as the identified subject object category, bounding box, generated subject image, and background image), render and display them on the front end, and support user confirmation or editing.

[0040] The application service layer dynamically schedules resources and executes AI inference based on front-end instructions. This includes adaptive preprocessing and intelligent scheduling and fault tolerance. Adaptive preprocessing involves squarening non-standard aspect ratio images and implementing a dynamic color differentiation mechanism to address recognition biases in visual models under special inputs. Intelligent scheduling and fault tolerance utilize a dynamic template selection mechanism based on metadata and historical quality scores, along with a closed-loop fault tolerance system that includes exponential backoff retries and automatic switching of alternative templates.

[0041] The application service layer is the core of the image processing method described above. This service layer can be further subdivided into a data access and adaptive processing layer, an intelligent scheduling and prompt management layer, a core inference and pipeline orchestration layer, and an asset management and quality assurance layer. The specific implementation process of each processing layer is described in detail below.

[0042] S11, Data Access and Adaptive Preprocessing Layer;

[0043] This layer is primarily responsible for standardizing the raw data and adapting it to the model, ensuring input quality.

[0044] (1) Adaptive resolution decision;

[0045] It is primarily used to address the issues that fixed-resolution processing cannot accommodate original images of different sizes, easily leading to problems such as small images being enlarged to fit the model or large images being compressed and losing details, and easily exceeding the model's processing limits. During processing, a scaling strategy is automatically matched based on the model's pixel thresholds (such as 512, 1024, and 2048), achieving a balance between detail preservation and model limitations.

[0046] (2) Image preprocessing;

[0047] It is mainly used to solve the problems of (1) visual illusion and inaccurate recognition. For example, when large models process non-square (non-standard ratio) images, they are prone to visual distortion and "illusion reading", which leads to deviation in reading the coordinates of visual anchor points; (2) visual conflict of visual anchor points: When the visual anchor points and background pixel colors are similar, large models are prone to visual confusion and cannot accurately locate or recognize the marks.

[0048] The improvements were primarily achieved through standardized square processing and a dynamic color differentiation mechanism. Standardized square processing targets images with non-standard aspect ratios, using a geometric filling strategy to convert them into standard proportions (such as squares), thus avoiding visual model feature distortion caused by abnormal aspect ratios. The dynamic color differentiation mechanism intelligently calculates the average RGB value of the area surrounding the visual anchor point when plotting at specified coordinates. If the Euclidean distance from the preset marker color is less than a threshold, the marker color is automatically switched to resolve visual confusion.

[0049] S12, Intelligent Scheduling and Prompt Management Layer;

[0050] This layer is primarily used for dynamic matching and version management of task instructions for image processing tasks, specifically including:

[0051] (1) Structured Template Library: This library addresses the issue of hard-coded Prompt messages in business logic, where modifications require a new release and version control is lacking, making A / B testing and performance tracking difficult. It resolves the high coupling between Prompt messages and business code, decoupling the message logic and enabling independent version management, while supporting dynamic filling at runtime.

[0052] (2) Intelligent Template Scheduler: Primarily used to address the issue that a single Prompt cannot adapt to all complex scenarios, and that the system lacks an automatic degradation and error correction mechanism when encountering long-tail requirements or model illusions, directly leading to task failure. It automatically filters primary / backup templates based on image metadata, task type, anchor point location, and historical quality data. It supports automatic degradation switching or triggering manual review when output fails to meet standards.

[0053] S13, Core Inference and Pipeline Orchestration Layer;

[0054] This layer is the business execution center of the overall architecture, responsible for complex image generation and segmentation tasks, specifically including:

[0055] (1) High Availability Model Client: This is mainly used to address the issue that large model API calls are greatly affected by network fluctuations, rate limiting, or occasional errors, and lack a refined retry strategy, which can easily lead to the interruption of the entire link. In solving this problem, the main approach is to encapsulate the parsing of large model requests and responses, and to build a strict JSON validation and parameterized retry mechanism (supporting linear backoff and preset number of retries) to ensure call stability.

[0056] (2) Layered pipeline scheduling: This is used to solve the problems of synchronous serial processing, long processing time, and low concurrency in complex tasks such as multi-subject splitting and background reconstruction; and the fact that a single point of failure will cause the global task to be invalidated. By constructing an end-to-end pipeline for whole image recognition, subject generation / background separation, cropping standardization, asset registration, and background reconstruction, parallel processing, failure rollback, and manual intervention are supported.

[0057] (3) Background separation and standardized cropping: This addresses the issue of harsh edges, color noise, and inconsistent size ratios after image cutout, which would result in obvious splicing marks if used directly in subsequent compositing. An adaptive edge feathering algorithm is used to smooth the cutout edges, eliminating jagged edges and noise, and standardized cropping is performed in conjunction with the original size information.

[0058] S14, Asset Management and Quality Assurance Layer.

[0059] This layer is used to ensure the traceability of data throughout the entire chain and the reliability of the output results, specifically including:

[0060] (1) Unified Asset Registration: This addresses issues such as the disconnect between generated materials and the original task context, the lack of a unified data dictionary, and the resulting fragmentation of contextual information during later retrieval, problem investigation, or secondary processing. It primarily ensures the traceability of assets throughout their entire lifecycle by resolving the lack of contextual association between generated materials and the original task.

[0061] (2) Quality Judgment and Rollback Strategy: This addresses the uncertainty of model output, the potential for a large number of incomplete or semantically incorrect scraps to flow downstream without automated quality inspection, and the increased cost of manual screening. A multi-dimensional quality verification mechanism (JSON integrity, semantic and spatial rationality) is established. When a judgment fails, automatic retry, alternative template switching, or redirection to manual review is implemented.

[0062] The data persistence layer is used to store static configurations, dynamic state data, and historical data, such as image resource storage, template and configuration storage (e.g., different versions of the Prompt template, static priority coefficients, resolution thresholds, context matching rules, and other configuration information).

[0063] To facilitate understanding, the following will be combined with... Figure 4 The overall flowchart shown further describes the above image processing method.

[0064] S402, User inputs image processing request;

[0065] S404 receives image processing requests or image processing tasks through the API controller and performs context binding;

[0066] For example, upon receiving a request from the front-end or upstream service, the system parses the session context (including seqID, userCode), task type identifier (taskType, such as visual anchor point localization or image layering), and image resource identifier (assetID). If the task involves spatial localization, the system synchronously receives the normalized coordinates (x, y) input by the user.

[0067] S406, through the input mapping business module, transforms the user's fuzzy or normalized interaction behavior on the front end into high-precision pixel-level instructions and context data that the back-end model can understand.

[0068] For example, when a user performs a momentary operation (such as clicking or dragging) on ​​a front-end image, it is usually necessary to map the obtained normalized coordinates relative to the image display area to pixel coordinates.

[0069] S408, Preprocessing of the original image;

[0070] This mainly includes adaptive resolution decision-making, coordinate mapping and edge protection, and image visual enhancement.

[0071] Among them, adaptive resolution is intended to resolve the contradiction between original images of different sizes and the fixed input size of the model, avoiding memory overflow due to excessively large images or loss of detail due to excessively small images; coordinate mapping and edge protection are intended to ensure that the normalized coordinates (x, y) of user clicks on the front end can accurately correspond to the processed image pixels, and to prevent anchor objects (such as people) from being truncated by the edges of the screen; image visual enhancement is used to solve the "visual illusion" problem of the model, that is, the anchor mark is similar in color to the background, causing the model to fail to recognize the mark, or non-square images cause distortion of model features.

[0072] S410, Preview Generation: This generates a standardized image file with clear visual markings that meets the model input requirements. This image file is used for model analysis.

[0073] S412, invoke the Prompt template decision model;

[0074] The Prompt template decision model does not directly receive the original image, but instead receives structured metadata from the preprocessing and API controller shown in step S408 above. This metadata constitutes the feature vector of the decision.

[0075] Based primarily on the contextual features of the current image task, the system dynamically filters, scores, and selects the optimal Prompt template (including primary and backup templates) from the template library to maximize the accuracy of AI generation / recognition and reduce illusions.

[0076] S414: Using the target processing model, following the prompts of the selected Prompt template and the preprocessed intermediate image, execute the image processing logic to obtain the target processing result; and register the target processing result as an asset.

[0077] The target processing results include, but are not limited to, the structured recognition results obtained from the visual anchor point localization (first type of image processing task) task, and the transparent background subject object image and the background image of each subject object obtained from the image layering (second type of image processing task).

[0078] It should be noted that the technical solution proposed in this application is an image processing method based on dynamic template driving. Its core lies in a reusable dynamic template scheduling mechanism, which can dynamically select and assemble the optimal prompt text template according to the task type and image context. Based on this core mechanism, this application provides two specific application examples: visual anchor point localization and image layering generation, thereby realizing a complete technical closed loop from precise interaction to automated production.

[0079] Both types of image processing tasks mentioned above are implemented based on a template-driven mechanism, and their core execution logic includes a general processing stage and an application branch stage.

[0080] The general processing stage is a necessary step for both types of image processing tasks, responsible for standard data access and intelligent instruction generation.

[0081] (1) Input parsing and metadata retrieval;

[0082] The system receives the session context, task type identifier (taskType), and resource identifier (assetID). If the task involves spatial positioning (such as anchor point positioning), normalized coordinates (x, y) are received synchronously. Then, the BizAsset service is called to obtain the raw image metadata (W0, H0) and file size.

[0083] (2) Resolution decision and adaptive preprocessing;

[0084] Introduce the following parameterized threshold control strategy:

[0085] (a) Downsampling: When W0×H0>maxPixels, perform proportional scaling (e.g., downscaling to 1024 or 512 pixels) to reduce computational load.

[0086] (b) Upsampling / Error reporting: For low-resolution images below the safety threshold, perform safety threshold upsampling or directly report an error to block them.

[0087] (c) Coordinate Mapping and Edge Protection (Location Task Only): Execute the high-precision mapping algorithm pixel_x = round(x×W) and introduce an edge protection mechanism. When the anchor point is close to the boundary (<5px), edge=true is automatically marked to prevent visual feature truncation and provide accurate pixel-level input for subsequent template selection.

[0088] (3) Dynamic template scheduling and high availability inference.

[0089] Template matching: Based on image metadata, task type, and anchor point location features, filter and sort matching Prompt templates from the template library.

[0090] Primary / backup fault tolerance strategy: Prioritize the selection of high-priority templates and pre-configure backup templates.

[0091] High availability retry mechanism: During the model invocation phase, an exponential backoff retry mechanism is introduced (initial delay of 4 seconds, maximum loop of 10 times) to cope with network fluctuations or momentary model unavailability and ensure a stable acquisition of a response containing a valid JSON structure.

[0092] The different application branches between the two types of image processing tasks will be described in detail below with specific real-time examples.

[0093] Using the embodiments provided in this application, by reading image metadata and preprocessing the original image, the non-standard aspect ratio original image is transformed into an intermediate image of the target size for the adaptive model, avoiding visual distortion caused by abnormal aspect ratio. Based on the task type identifier, the spatial distribution information of the intermediate image is extracted to form anchor point position features, achieving precise mapping from user interaction coordinates to pixel-level positions. Combining image metadata, task type, and anchor point position features, target text templates are dynamically selected from a preset template library, ensuring a high semantic fit between the Prompt and the current image processing task and anchor point position, suppressing semantic illusions caused by instruction mismatch. This allows the target processing model to accurately execute image processing logic matching different processing tasks under the prompt of the target text template. In other words, by using dynamically adapted constraints to guide model generation, while ensuring positioning accuracy, the structure, semantic accuracy, and geometric consistency of the generated results are improved, solving the technical problems of large positioning deviation, uncontrollable generation, and rigid processes in traditional solutions, thus achieving the technical effect of improving image processing quality.

[0094] As an optional example, when the task type identifier indicates that the image processing task is for visual anchor point localization, the above method includes: obtaining the anchor point coordinates when a click operation is performed on the original image from the image processing task; mapping the anchor point coordinates to actual pixel coordinates; determining the anchor point position features of the visual anchor point based on spatial distribution information when it is determined based on the actual pixel coordinates that the visual anchor point is not located at the image edge of the middle image; and dynamically selecting target text templates from a preset template library based on image metadata, task type identifier, and anchor point position features.

[0095] At the front-end interaction level, users typically perform clicks (or other instantaneous operations) on a view that has been zoomed or stretched. The coordinates obtained at this time are usually normalized coordinates (x, y) relative to the current view window. That is, what is obtained is the raw interaction data at the moment the user performs the click operation. The core issue is to solve the semantic alignment problem between user interaction data and back-end processed data.

[0096] Since the images displayed on the front end are usually adaptively scaled to fit different screen sizes, and large models usually need to be input at a standard size with a specific resolution to achieve the best performance, it is necessary to establish a mapping and transformation from the user interaction coordinate system to the pixel coordinate system.

[0097] This embodiment constructs a high-fidelity mapping algorithm by combining the width and height metadata of the original image with the size of the intermediate image currently used for model analysis. Specifically, a scaling factor is calculated to convert the normalized coordinates into the actual pixel coordinates of the intermediate image. This process not only solves the coordinate offset problem caused by image scaling but also ensures that the visual anchor point in the model input image accurately falls on the pixel position of the target object (i.e., the object or main subject), thereby avoiding recognition bias caused by coordinate offset, that is, preventing the model from misinterpreting the object the user intends to point to.

[0098] In traditional approaches, if the visual anchor point is close to the image edge, the target object is often truncated, leading to model recognition failure or inaccurate bounding box prediction. This embodiment dynamically determines whether the visual anchor point is in an edge region by predicting the distance between the visual anchor point and the image boundary. If it is not in an edge region, structured anchor point position features are further extracted based on spatial distribution information. These features are not simple coordinate values, but spatial labels that have undergone semantic abstraction. This processing method allows the subsequent template scheduler to understand the spatial context of the anchor point, rather than just processing a numerical point, thus providing a basis for selecting a prompt template that better fits the spatial layout and improving the accuracy of the model's perception of the integrity of the subject.

[0099] It should be noted that for the visual anchor point localization task, the input for edge protection is a single point (the user's click location). By detecting whether this point is close to the image edge, a boolean flag, such as edge=true or false, is output. For the image layering and decomposition task, the input is multiple bounding boxes (generated by performing global analysis on the intermediate image to identify various subject objects). The input is the edge marker assigned to each subject object, is_edge_subject=true or false, which is detected as the edge marker of the image edge.

[0100] Instead of using a single, fixed prompt, the system constructs a multi-dimensional decision-making model. Inputs include image metadata reflecting the image's physical properties, a task type identifier clearly indicating the business intent, and the aforementioned extracted anchor point location features. Using this data, matching calculations are performed within a pre-defined structured template library. For example, if a visual anchor point is identified as being located in the center of the image and the image is high-resolution, templates optimized for central subject recognition are prioritized; if the image has a unique aspect ratio, prompt templates adapted to that ratio are selected.

[0101] Through the dynamic filtering described above, a target text template that is highly constrained and less prone to semantic illusion can be automatically assembled based on the specific scene and visual anchor point position of the current image. This mechanism effectively reduces the risk of illusion when large models process complex backgrounds or non-standard compositions, ensuring the structured and highly reliable nature of the visual anchor point localization results.

[0102] As an optional implementation, the above method dynamically selects target text templates from a preset template library based on image metadata, task type identifiers, and anchor point location features, including:

[0103] Based on image metadata, task type identifiers, and anchor point location features, an initial text template is selected from a preset template library.

[0104] The target text template is obtained by filling the placeholders in the initial text template with the multimodal image data extracted based on the anchor point location features and the task context.

[0105] Multimodal image data includes, but is not limited to, semantic description data, spatial location data, and context / environment data.

[0106] In other words, the template scheduling module retrieves the corresponding template based on the current task type, resolution, and edge flag (i.e., whether edge equals true), replaces the placeholders, and generates the final finalPrompt template.

[0107] As another optional implementation, the target text template and intermediate image are input into the target processing model to execute image processing logic corresponding to the task type identifier, including: calling the target processing model based on a preset retry strategy until a response result containing compliant structured data is obtained or the maximum number of retries is reached; parsing the target object information in the response result, wherein the target object information includes object type, bounding box coordinates and confidence level; adding the template identifier of the target text template, the current loop count and the image resource identifier of the original image to the image metadata to obtain the structured recognition result.

[0108] The confidence score mentioned above represents the reliability rating of the structured recognition result, while the bounding box provides the precise location of the object in the image.

[0109] To more clearly understand the processing flow of visual anchor point localization (Type I image processing task), the following section will combine... Figure 5 The overall processing flow for visual anchor point positioning is described in detail.

[0110] S502, request reception is bound to context;

[0111] The front end or server end receives the normalized anchor coordinates (x, y) input by the user and the incoming image resource ID (assetID), and binds the session context containing seqId and userCode at the same time to ensure the traceability of the task link.

[0112] S504: analyze and dynamically parse the image size of the original image;

[0113] Read the metadata (original width W0, original height H0) of the target resource, and dynamically calculate the target size (W, H) actually used for model analysis according to the preset maxPixels / minPixels threshold rule.

[0114] S506: multi-dimensional coordinate mapping calculation;

[0115] That is, a high-precision mapping algorithm is executed to convert the normalized coordinates into actual pixel coordinates pixel_x=round(x×W) and pixel_y=round(y×H).

[0116] Simultaneously calculate the thumbnail mapping coordinates (thumbnailX / thumbnailY) to ensure precise alignment at the UI layer and original image traceback. If the calculated pixel_x < marginPx (e.g., 5px) or > W-marginPx, edge=true is marked. This mark will be passed to the template scheduling stage to trigger a dedicated edge template and prevent the subject from being truncated.

[0117] S508: image preprocessing and cloud storage;

[0118] Draw an anchor mark on the original image or the scaled image, and crop a square preview area according to the Edge Protection mechanism. The processed image is uploaded to the temporary OSS storage to generate a previewUrl.

[0119] S510: dynamic assembly of Prompt templates;

[0120] The template scheduling module retrieves the corresponding template according to the current task type, resolution and edge flag, and replaces the placeholder to generate the final finalPrompt.

[0121] S512: model invocation and fault-tolerant retry;

[0122] Invoke the model client, use the encapsulated analyze Image Retry method (with built-in exponential backoff policy and maximum retry count) to obtain the model return, and verify the strict JSON format to ensure that a valid response containing the objects or items field is obtained.

[0123] S514, Result Parsing and Metadata Registration.

[0124] The returned JSON data (containing analysis and objects) is parsed, and the template ID, loop count (loopNum), and original assetID are appended to the metadata (appendInfo). Finally, a structured recognition result is returned, which is used to describe the visual anchor information, specifically for accurately locating the visual anchor.

[0125] As an optional example, when the task type identifier indicates that the image processing task is a hierarchical object splitting, target text templates are dynamically selected from a preset template library based on image metadata, task type identifier, and anchor point location features. This includes: performing global parsing on the intermediate image to obtain a list of object information; selecting multiple candidate text templates from the preset template library based on image metadata, task type identifier, and object information; determining a comprehensive score for each candidate text template based on static priority, historical success rate, and context matching degree; ranking the multiple candidate text templates according to the comprehensive score, and determining the main text template and backup text templates based on the ranking result. The context matching degree is determined based on the degree of matching between the object information and the preset conditions of the template.

[0126] Each item in the object information list includes the object name, canvas position, and object type.

[0127] In the global parsing stage, instead of relying solely on simple image classification, a large-scale vision-language model is used to perform deep semantic understanding and structured extraction on the preprocessed intermediate images. The aim is to transform unstructured visual information into a list of object information that the model can understand. Specifically, the model identifies individual subjects (such as people, products, and text) or individual objects in the image and accurately outputs the name (semantic label), the relative position coordinates on the canvas (for subsequent anchor point mapping), and the object type (such as people, animals, and decorations) for each object. The results of global parsing form the basis for refined template matching, providing fine-grained contextual features for subsequent template selection.

[0128] In the candidate template selection stage, a multi-dimensional filtering mechanism was constructed. Unlike traditional methods based on single keyword matching, this embodiment combines static image metadata (such as resolution and aspect ratio), the current task type identifier (explicitly indicating an image layering and splitting task), the canvas position features obtained in the previous step, and the object information itself. Through the rule engine, the system first removes templates incompatible with the current scene from the preset template library, thereby quickly narrowing the search space and improving retrieval efficiency.

[0129] To overcome the limitations of single-index selection, this embodiment proposes a dynamic scoring model based on a weighted formula. This model comprehensively considers three key dimensions: first, static priority, reflecting the template's fundamental importance in the business logic, which is only fine-tuned when long-term trends change; second, historical success rate, i.e., the template's actual performance score in similar past scenarios, updated in real-time using methods such as exponential moving averages to ensure high availability; and third, context matching, a dynamically calculated real-time indicator, further broken down into topic consistency, size matching, anchor layout matching, and placeholder filling completeness. By introducing coefficients α, β, and γ, and weighting and summing the scores of the above three dimensions based on these three coefficients, a comprehensive score for each candidate text template is obtained, accurately quantifying the suitability of each candidate template for the current specific task.

[0130] Candidate templates are ranked based on their comprehensive scores. The template ranked first with a score exceeding a threshold is selected as the primary text template for the initial call to the generation model, aiming for optimal generation results. Templates with the next highest scores that meet basic usability requirements are designated as backup or alternative text templates. This primary / backup design, combined with the exponential backoff retries and automatic degradation mechanisms mentioned in the above embodiments, constitutes a highly available fault-tolerant mechanism.

[0131] For example, when the main template fails to meet the quality standards due to network fluctuations, model illusions, or substandard generated results, the system can automatically switch to the backup text template. This improves the success rate and consistency of image layering generation tasks while ensuring processing efficiency, and solves the technical problem of uncontrollable generated results in complex multi-subject scenarios.

[0132] As an optional implementation, the above-mentioned input of the target text template and intermediate image into the target processing model to execute image processing logic corresponding to the task type identifier includes: dynamically filling the placeholders in the main text template with the object name and canvas position to obtain the target main text template; by inputting the target main text template and intermediate image into the target processing model, extracting the object images of each object in the object information list from the intermediate image; making the background of the object image transparent, and cropping the object image with the transparent background to obtain the target object image.

[0133] In the dynamic template assembly stage, instead of simply invoking fixed instructions, the Prompt is personalized through a data-driven approach. The structured object information extracted in the above embodiments (including specific object names, precise canvas coordinates, and types) is dynamically injected into specific placeholders in the main text template. This mechanism solves the problem that traditional hard-coded Prompts are difficult to adapt to complex and ever-changing scenarios, enabling the generated instructions to precisely constrain the model to focus on specific main objects. For example, if the template requires recognizing a person, the entered name and location coordinates will guide the multimodal model to ignore background noise and focus on the semantic understanding of that specific area, thereby reducing model illusions and ensuring the accuracy and consistency of the extracted content.

[0134] In the main body segmentation and generation stage, the system uses a fully filled target main text template and a preprocessed intermediate image as input to call a target processing model (such as a large model that supports visual anchor point localization) to perform a refined image analysis task. Its core lies in utilizing the strong semantic constraints provided by dynamic templates, enabling the model to output a structured JSON response containing object bounding boxes, confidence scores, and semantic labels. Based on this, the system accurately locates and crops the original image regions of each independent object from the entire image. This process achieves decoupling from the semantics of the entire image to local objects, laying the data foundation for subsequent independent processing, while ensuring the traceability of the generated results through structured output.

[0135] To eliminate issues such as harsh edges, unwanted colors, or residual background in the cutout image, an adaptive edge feathering algorithm and transparency processing techniques were introduced. Specifically, the background of the extracted object image is first removed, generating a transparent background image with an alpha channel to preserve subject details and remove irrelevant background interference. Then, combining the original size information and standardized cropping rules, the transparent background object image undergoes secondary cropping and size alignment, removing unnecessary blank areas to ensure the aspect ratio of the output image meets downstream business requirements (such as standard specifications for e-commerce main images or social media images). Through unified data format encapsulation, a connection is established between the generated materials and the original task context, facilitating subsequent asset management and secondary processing, and achieving high-quality, standardized object asset output.

[0136] As another optional implementation, the above method also includes: obtaining background images of each object by inputting the target main text template and intermediate images into the target processing model; and determining the target object image and background image as the target processing result.

[0137] The target main text template is not a static, fixed prompt, but a structured prompt optimized by a dynamic scheduling layer. First, based on image metadata obtained in the preprocessing stage (such as subject type, edge protection markers, scene context, etc.), a template scheduling decision model calculates the comprehensive score of each candidate template, thereby determining the most suitable main text template for the current task scenario. This template not only contains semantic descriptions of background style and lighting atmosphere, but also implicitly contains constraints on the spatial relationships of the subject objects, aiming to guide the generative model (i.e., the target processing model) to maintain natural integration and semantic consistency with the subject edges when completing the background (i.e., generating background images for each object).

[0138] Intermediate images can refer to, but are not limited to, standardized subject images generated after the subject object processing stage in the layered pipeline. Specifically, this image has undergone intelligent background separation, adaptive edge processing, and standardized cropping, avoiding the jagged edges and noise common in original image matting, ensuring the clear outline and independent asset attributes of the subject object. The optimized target main text template and the processed intermediate image are input together into the target processing model (such as an image generation network based on a diffusion model). The model can understand the visual features of the subject object and the compositional intent of the text description, thereby regenerating background images with reasonable perspective effects, i.e., background images of each object. The generated background images and the corresponding target object images are determined as the target processing results.

[0139] The above method not only achieves decoupled generation of the main object and the background, ensuring high-fidelity preservation of the main object's details, but also effectively suppresses semantic illusions in the generated model through the constraint of dynamic templates, ensuring the overall visual coordination and adaptability of the synthesized results to the business scenario, thus completing an end-to-end closed loop from single-point recognition to complete scene reconstruction.

[0140] As another optional implementation, the above method also includes at least one of the following: automatically switching to a backup text template when the main text template fails to be called or the image quality of the target object image does not meet the standard; marking the image processing task as pending review when the backup text template fails to be called.

[0141] In high-concurrency and complex scenarios of image generation, large model API calls are subject to uncertainties such as network fluctuations or occasional semantic illusions. To address this, this embodiment constructs a closed-loop automatic fault-tolerance system. When the main text template call fails (e.g., timeout, return of an invalid JSON structure, or HTTP error) or the generated target image is deemed substandard by multi-dimensional quality checks (including JSON integrity, semantic rationality, and spatial rationality), the system does not immediately terminate with an error report but triggers a degradation strategy. At this point, the scheduler automatically retrieves a pre-set backup text template. This backup template typically has different Prompt constraint strategies or is optimized for specific long-tail scenarios. The system re-inputs it along with the intermediate image into the model for retrying. If the backup template is successfully generated and passes quality checks, the result is directly output, thereby improving the success rate of a single task while ensuring system stability.

[0142] In addition, for extreme cases, such as when the backup text template also fails to be invoked or the generated results still fail to meet the quality threshold, a manual intervention mechanism is implemented. At this time, the image processing task is suspended and its current status is marked as pending manual review. During manual review, the image processing task, related image data, failure logs, and the template version used are fully recorded and transferred to the manual review queue.

[0143] This not only prevents invalid data from flowing into downstream business processes and reduces noise in automated processes, but also provides real, high-quality labeled data for subsequent optimization of model parameters, adjustment of template weights, or repair of business logic.

[0144] To more clearly understand the process of image layering and subject segmentation, the following will combine... Figure 6 The overall implementation process of this type of image processing task is further described. Figure 6 An automated image editing pipeline was built, encompassing whole-image analysis, subject segmentation, asset standardization, and background generation. Specifically, it includes:

[0145] S602, Image processing task request reception and context binding;

[0146] The front-end or server receives the image resource (assetID) selected by the user and binds it to the session context containing seqID and userCode to ensure the traceability of the task chain.

[0147] S604, dynamic analysis of dimensions;

[0148] Read the target resource's metadata (original width W0, height H0), and dynamically calculate the actual target size (W, H) used for AI analysis based on the preset maxPixels / minPixels threshold rules.

[0149] S606, global subject recognition for the entire image;

[0150] At this point, fixed prompts are used to perform global analysis and recognition of the image. For example... Figure 6 As shown, the dynamic template decision model (task type: parsing all subjects) is invoked to perform a global analysis of the entire image, returning a list of Image SubjectPojos, and accurately extracting the name, canvas position (position_on_canvas) and type of each subject object.

[0151] S608, detachable main body generation;

[0152] like Figure 6 The parallel processing queue shown above, for separable subjects, dynamically injects subject name and location information, and calls a dynamic template decision model (task type: subject splitting) to obtain the image address of a single subject object. The system can intelligently switch splitting constraints (such as preserving contact surfaces or preserving integrity) based on the subject category (person / item / text).

[0153] S610, main body back-cutting and asset standardization;

[0154] Download the generated main image, call the background separation service to remove excess white space, and generate a standardized main image. Then upload it to OSS and register it as an independent asset, setting appendInfo={"resource_asset_id": original assetID} to preserve the lineage.

[0155] S612, background generation for each main object;

[0156] like Figure 6 The task type shown is the background image generation part. It calls the dynamic template decision model (task type: generate background image), outputs the matching background image, and synchronously registers it as a background asset. It sets appendInfo = {"resource_asset_id": original assetID} to retain the association relationship.

[0157] S614, Result Encapsulation and Audit Traceability.

[0158] A unified AiEditImagePojo object is returned, containing bgAssetId, bgUrl, and a subject[] array (containing the assetID, thumbnailX / Y, originUrl, etc. of each subject object). Simultaneously, the template version and task execution log are recorded, providing complete traceability evidence for subsequent auditing and version rollback.

[0159] Based on the above analysis, it can be seen that the core execution logic of the two types of image processing tasks—visual anchor point localization and image layer generation / subject segmentation—includes a general processing flow and an application branch flow. The general processing flow has been described in the above embodiments; the application branches for the two types of image processing tasks are described below.

[0160] Application branch one is the visual anchor point localization process. When the task type is anchor point localization, the following unique process is executed:

[0161] (1) Visual marker generation: Based on the calculated pixel coordinates, visual anchor points are drawn on the preprocessed image, and the marker style is adjusted in combination with edge protection markers;

[0162] (2) Location result parsing: Parse the JSON returned by the model, extract the category, bounding box and confidence of the object pointed to by the anchor point, and encapsulate it into a location result object and return it.

[0163] Application branch two is the image layering generation process. When the image processing task type is image layering, the following process is executed:

[0164] (1) Parallelize the generation pipeline;

[0165] For the identified multiple subjects or multiple subject objects, the system adopts a queue-level parallel strategy to independently execute the following sub-tasks:

[0166] Main processing: Generate, intelligent background separation, crop, and edge feathering are performed sequentially.

[0167] Transaction isolation: If a single entity fails to process a transaction, a retry is automatically triggered or the transaction is marked for manual review. Transactions between entities are isolated from each other and do not affect each other.

[0168] (2) Parallel background or background image generation;

[0169] Call the background generation template and then call the generation model to complete the background asset registration.

[0170] (3) Three-dimensional quality verification and background reconstruction.

[0171] Three-dimensional quality verification: A rigorous quality assessment system is introduced, including JSON integrity verification (field missing detection), semantic reasonableness (whether the content conforms to the Prompt), and spatial reasonableness (whether the generated size is distorted). If any indicator fails to meet the standard, the backup template rollback mechanism will be automatically triggered.

[0172] After processing according to the general process described above, application branch one, and application branch two, different image processing results (which can also be understood as assets) are obtained. At this point, it is also necessary to consider encapsulating and persisting the assets, as detailed below:

[0173] (1) Structure encapsulation: Encapsulate the generated background asset ID, the subject (or subject object, object) list and the original URL into a standard object for return.

[0174] (2) Data closed loop: Completely record task execution logs, template invocation versions and asset lineage information to form a complete closed loop of business and data.

[0175] In addition, the technical solution of the present application also uses the following rule logic:

[0176] (1) Coordinate mapping and scaling rule;

[0177] To ensure that the analysis result of the AI model can be accurately traced back to the original image, the following mapping and protection mechanism is adopted:

[0178] The basic mapping formula is pixel_x=round(x×W), pixel_y=round(y×H).

[0179] Thumbnail tracing logic: When the original image is compressed (W0>W), the scaling ratio r=W / W0 is calculated. The system needs to save the original coordinate originalX=round(pixel_x / r), so that subsequent operations can accurately locate the position in the original image.

[0180] Edge protection mechanism: If the calculated pixel_x<marginPx (e.g., 5px) or >W-marginPx, mark edge=true. This mark will be passed to the template scheduling stage to trigger the dedicated edge template and avoid the subject being truncated.

[0181] (2) Resolution decision strategy;

[0182] The threshold is parametrically controlled through the configuration file to dynamically determine the analysis size of the AI model:

[0183] High resolution downgrade: If W0×H0>maxPixels, preferentially scale to targetW=1024, and calculate targetH proportionally; if the limit is still exceeded, fall back to targetW=512 (or 2048 as a special fallback value).

[0184] Low resolution protection: If W0×H0<minPixels, an exception of excessively low resolution is fed back by default; an optional strategy is to upsample to a safe threshold on the premise of maintaining the quality.

[0185] (3) Template scheduling decision model.

[0186] As Figure 7As shown in the embodiments of this application, a rule-based and priority-based Prompt template decision-making mechanism is provided, as follows:

[0187] Input feature extraction: Collect candidate sets of imageMeta, taskType, anchorPos (edge / center), and subjectType.

[0188] Rule filtering: Filter out the set of templates that meet the input feature conditions.

[0189] Priority sorting: Sort in descending order by priority and historical success rate score.

[0190] Primary / backup selection strategy: Select the first element in the sorted list as the primary template. Extract the second element as the backup.

[0191] Runtime fault tolerance: If the primary call fails or the quality check fails, it automatically switches to backup; if the backup template fails, the task is marked as manually reviewed.

[0192] The comprehensive score of the candidate text template can be calculated, but is not limited to, by the following formula (1):

[0193] Score=α×StaticPriority+β×HistoricalScore+γ×ContextMatch (1)

[0194] Among them, StaticPriority is the static priority, HistoricalScore is the historical success rate, and ContextMatch is the context matching degree (such as size matching, topic matching). ContextMatch first uses fixed prompts to determine the topic of the original image, and then calculates the topic matching degree.

[0195] The initial values ​​of the weighting coefficients are α=0.35, β=0.35, and γ=0.35.

[0196] The initial values ​​of each parameter are StaticPriority=0.5, with a value range of [0, 1.0], HistoricalScore=0.5, with a value range of [0, 1.0], and ContextMatch=0.5, with a value range of [-0.2, 1.0].

[0197] The update logic for each of the above parameters is as follows:

[0198] StaticPriority is a relatively fixed priority derived from historical data statistics. It is not adjusted based on a single success or failure, but only when a trend emerges in continuous performance. The current strategy is to set StaticPriority_new = min(StaticPriority_old + 0.02, 1.0) after 20 consecutive successes. After two consecutive failures, set StaticPriority_new = max(StaticPriority_old - 0.02, 0).

[0199] HistoricalScore is a historical score reflecting the template's recent performance and is responsible for the result of a single request. On success, HistoricalScore_new = min(HistoricalScore_old + 0.06, 1.0). On failure, HistoricalScore_new = max(HistoricalScore_old - 0.10, 0).

[0200] ContextMatch calculates the context matching degree between the current task and the template in real time. ContextMatch = themeMatch + sizeMatch + anchorMatch + placeholderCoverage.

[0201] The main meanings and value ranges are as follows:

[0202] themeMatch: The consistency between the task theme and the template theme. Value range [-0.2, 0.3], default value 0.15. Value rules: exact theme matching, value is 0.3; partially relevant, value is 0.15; irrelevant, value is 0; conflicting / obvious mismatch, value is -0.2.

[0203] `sizeMatch` represents the degree of match between the current material / output size and the expected size of the template. The value range is [-0.2, 0.2], with a default value of 0.1. Value rules: High match = 0.2, Acceptable = 0, Obvious mismatch = -0.2.

[0204] `anchorMatch`: The degree to which the anchor point position matches the template design. The value range is [0.0, 0.2], with a default value of 0.1. Value rules: 0.2 for a match to the expected anchor point layout, 0.1 for an approximate match, and 0 for no match.

[0205] `placeholderCoverage` determines the completeness of the template placeholder / parameter's fill in the current scene. Its value ranges from -0.3 to 0.3, with a default value of 0.225. It is calculated as `placeholderCoverage = (filledCount / totalCount) × 0.3`. The rule is that complete fill equals 0.3, while partial fill, for example, equals 0.3. =>0.225, if the placeholder for the required field is missing, the value is -0.3.

[0206] like Figure 7 As shown, after calculating the comprehensive score of each candidate template in the above manner, the first one in the ranking is selected as the main text template, and the second one is extracted as the alternative text template (i.e., the candidate template).

[0207] Runtime fault tolerance: If the main text template fails to be called or the quality check fails, it will automatically switch to the backup text template; if the backup template fails to be called, the task will be marked as manually reviewed.

[0208] As can be seen from the description of the above embodiments, the technical solution of this application aims to solve the technical problems existing in the current AIGC image editing technology, such as low positioning accuracy, uncontrollable generation results, and fragmented process. Specific objectives include:

[0209] (1) Achieve pixel-level precise positioning and structured recognition under user-specified normalized coordinates, reduce model illusion and improve recognition credibility;

[0210] (2) Establish an end-to-end automated processing link from whole image recognition, subject splitting, background separation to background reconstruction to solve the processing efficiency and consistency problems in multi-subject scenarios.

[0211] (3) By using a dynamic Prompt template library and a scenario-based selection mechanism, the constraints, versionability, and rollback of the prompt are guaranteed, reducing generation risks and facilitating quality control and A / B optimization. This solves the problem that in the existing technology, a single Prompt template is difficult to adapt to complex and ever-changing scenarios, resulting in semantic illusions in the model and uncontrollable generated content.

[0212] In other words, to address the pain points of multimodal models in image editing tasks, such as large localization errors, uncontrollable generation results, and low processing efficiency, this application's technical solution further introduces resolution adaptive decision-making, a high-availability fault-tolerant strategy based on template degradation, and a multi-dimensional quality verification mechanism. Through these technical means, the shortcomings of traditional methods are effectively overcome, achieving highly available and traceable image structured decomposition and background generation.

[0213] Compared with traditional technical solutions, the technical solution of this application has at least the following beneficial effects:

[0214] (1) Pixel-level positioning accuracy: Through coordinate mapping and thumbnail alignment mechanism, pixel-level accurate positioning is ensured at different resolutions;

[0215] (2) Structured recognition and generation: The templated prompt scheduling mechanism effectively reduces model illusion and improves the structured nature of the results;

[0216] (3) Automated pipeline: End-to-end layered main body splitting and generation process, which greatly improves processing efficiency and consistency;

[0217] (4) Quality controllability: A sound quality judgment and rollback mechanism ensures the reliability of the generated results;

[0218] (5) Traceability: Complete asset metadata and lineage management facilitate subsequent auditing and optimization.

[0219] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0220] According to another aspect of the embodiments of this application, as follows is also provided Figure 8 An image processing apparatus is shown, comprising: a first acquisition unit 802, configured to acquire a task type identifier and an image resource identifier in response to a received image processing task; a reading unit 804, configured to read image metadata of the original image based on the image resource identifier; a preprocessing unit 806, configured to preprocess the original image to obtain an intermediate image of the target size; and extract spatial distribution information of each object in the intermediate image based on the task type identifier to form anchor point position features; a filtering unit 808, configured to dynamically filter target text templates from a preset template library based on image metadata, task type identifier, and anchor point position features; and a first processing unit 810, configured to input the target text template and the intermediate image into a target processing model to execute image processing logic corresponding to the task type identifier and obtain the target processing result.

[0221] Optionally, when the task type identifier indicates that the image processing task is visual anchor point localization, the above device further includes: a second acquisition unit, used to acquire the anchor point coordinates when performing a click operation on the original image from the image processing task; a mapping unit, used to map the anchor point coordinates to actual pixel coordinates; a second processing unit, used to determine the anchor point position features of the visual anchor point based on spatial distribution information when it is determined based on the actual pixel coordinates that the visual anchor point is not located at the image edge of the intermediate image; and a filtering unit, used to dynamically filter target text templates from a preset template library based on image metadata, task type identifier, and anchor point position features.

[0222] Optionally, the above-mentioned filtering unit includes: a first filtering module, used to filter out an initial text template from a preset template library based on image metadata, task type identifier and anchor point position features; and a first processing module, used to fill the placeholders in the initial text template with multimodal image data extracted based on anchor point position features and task context to obtain a target text template.

[0223] Optionally, the first processing unit 810 includes: a first invocation module, used to invoke the target processing model based on a preset retry strategy until a response result containing compliant structured data is obtained or the maximum number of retries is reached; a first parsing module, used to parse the target object information in the response result, wherein the target object information includes object type, bounding box coordinates and confidence level; and an addition module, used to add the template identifier of the target text template, the current loop count and the image resource identifier of the original image to the image metadata to obtain the structured recognition result.

[0224] Optionally, when the task type identifier indicates that the image processing task is hierarchical object splitting, the filtering unit 808 further includes: a second parsing module, used to perform global parsing on the intermediate image to obtain a list of object information; a second filtering module, used to filter multiple candidate text templates from a preset template library based on image metadata, task type identifier, and object information; a second processing module, used to determine the comprehensive score of each candidate text template based on static priority, historical success rate, and context matching degree; and a sorting module, used to sort the multiple candidate text templates according to the comprehensive score, and determine the main text template and backup text templates according to the sorting result.

[0225] Optionally, the first processing unit 810 further includes: a filling module, used to dynamically fill the placeholders in the main text template with the object name and canvas position to obtain the target main text template; a splitting module, used to input the target main text template and the intermediate image into the target processing model, and split the object images of each object in the object information list from the intermediate image; and a transparency processing module, used to make the background of the object image transparent and to crop the object image with the transparent background to obtain the target object image.

[0226] Optionally, the above apparatus further includes: a third processing unit, configured to perform at least one of the following: automatically switching to a backup text template if the main text template fails to be invoked or the image quality of the target object image does not meet the standard; and marking the image processing task as pending review if the backup text template fails to be invoked.

[0227] It should be noted that the embodiments of the image processing apparatus described here can refer to the embodiments of the image processing method described above, and will not be repeated here.

[0228] According to another aspect of the embodiments of this application, an electronic device for implementing the above-described image processing method is also provided. This electronic device may be... Figure 1 The target terminal or server is shown. This embodiment uses the electronic device as an example to illustrate the concept. Figure 9 As shown, the electronic device includes a memory 902 and a processor 904. The memory 902 stores a computer program, and the processor 904 is configured to execute the steps of any of the above method embodiments through the computer program.

[0229] Optionally, the aforementioned electronic device may be located in at least one of a plurality of network devices of the computer.

[0230] Optionally, the processor can be configured to execute the processing procedures shown in steps S202 to S210 above via a computer program.

[0231] Alternatively, as those skilled in the art will understand, Figure 9 The structure shown is for illustrative purposes only. Figure 9 This does not limit the structure of the aforementioned electronic devices or electronic equipment. For example, electronic devices or electronic equipment may also include components that are more... Figure 9 The more or fewer components shown (such as network interfaces, etc.), or having the same Figure 9 The different configurations shown.

[0232] The memory 902 can be used to store software programs and modules, such as the program instructions / modules corresponding to the image processing method and apparatus in this embodiment. The processor 904 executes various functional applications and data processing by running the software programs and modules stored in the memory 902, thereby implementing the aforementioned image processing method. The memory 902 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 902 may further include memory remotely located relative to the processor 904, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Specifically, the memory 902 may be used, but is not limited to, to store task type identifiers, image resource identifiers, and image metadata, etc. As an example, such as... Figure 9 As shown, the memory 902 may include, but is not limited to, the first acquisition unit 802, the reading unit 804, the preprocessing unit 806, the filtering unit 808, and the first processing unit 810 in the image processing device. Furthermore, it may include, but is not limited to, other module units in the image processing device, which will not be elaborated upon in this example.

[0233] Optionally, the transmission device 906 described above is used to receive or send data via a network. Specific examples of the network described above may include wired networks and wireless networks. In one example, the transmission device 906 includes a Network Interface Controller (NIC), which can be connected to other network devices and a router via a network cable to communicate with the Internet or a local area network. In another example, the transmission device 906 is a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0234] In addition, the above-mentioned electronic device also includes: a display 908 for displaying the original image and the transparent background image of each object obtained after the image processing task is completed; and a connection bus 910 for connecting the various module components in the above-mentioned electronic device.

[0235] In other embodiments, the target terminal or server described above can be a node in a distributed system. This distributed system can be a blockchain system, formed by connecting multiple nodes through network communication. The nodes can form a point-to-point network, and any type of computing device, such as a server or target terminal, can become a node in the blockchain system by joining this point-to-point network.

[0236] According to another aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the image processing method provided in various optional implementations of the above-described server verification processing, wherein the computer program is configured to execute the steps in any of the above-described method embodiments at runtime.

[0237] Optionally, in this embodiment, the computer-readable storage medium described above may be configured to store a computer program for executing steps S202 to S210 as described above.

[0238] Optionally, in embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0239] Optionally, in this embodiment, those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the target terminal. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0240] The sequence numbers of the embodiments in this application are merely for description and do not represent the superiority or inferiority of the embodiments. If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods in the various embodiments of this application.

[0241] In the above embodiments of this application, the descriptions of each embodiment have their own emphasis. Parts not described in detail in a certain embodiment can be referred to in the relevant descriptions of other embodiments. It should be understood that the disclosed client can be implemented in other ways in the several embodiments provided in this application. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection of units or modules may be electrical or other forms.

[0242] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units described above can be implemented in hardware or as software functional units.

[0243] The above are merely preferred embodiments of this application. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. An image processing method, characterized in that, include: In response to a received image processing task, obtain the task type identifier and image resource identifier; Based on the image resource identifier, read the image metadata of the original image; The original image is preprocessed to obtain an intermediate image of the target size; and based on the task type identifier, the spatial distribution information of each object in the intermediate image is extracted to form anchor point position features; Based on the image metadata, the task type identifier, and the anchor point location features, target text templates are dynamically selected from a preset template library; The target text template and the intermediate image are input into the target processing model to execute the image processing logic corresponding to the task type identifier and obtain the target processing result.

2. The method according to claim 1, characterized in that, When the task type identifier indicates that the image processing task is visual anchor point localization, the method includes: Obtain the anchor point coordinates when performing a click operation on the original image from the image processing task; Map the anchor point coordinates to actual pixel coordinates; If, based on the actual pixel coordinates, it is determined that the visual anchor point is not located at the image edge of the intermediate image, the anchor point position feature of the visual anchor point is determined based on the spatial distribution information. Based on the image metadata, the task type identifier, and the anchor point location features, target text templates are dynamically selected from a preset template library.

3. The method according to claim 2, characterized in that, The step of dynamically selecting target text templates from a preset template library based on the image metadata, the task type identifier, and the anchor point position features includes: Based on the image metadata, the task type identifier, and the anchor point location features, an initial text template is selected from a preset template library; The target text template is obtained by filling the placeholders in the initial text template with the multimodal image data extracted based on the anchor point location features and the task context.

4. The method according to claim 2, characterized in that, The target text template and the intermediate image are input into the target processing model to execute image processing logic corresponding to the task type identifier, including: Based on the preset retry strategy, the target processing model is invoked until a response result containing compliant structured data is obtained or the maximum number of retries is reached. Parse the target object information in the response result, wherein the target object information includes object type, bounding box coordinates, and confidence level; The template identifier of the target text template, the current loop count, and the image resource identifier of the original image are added to the image metadata to obtain the structured recognition result.

5. The method according to claim 1, characterized in that, When the task type identifier indicates that the image processing task is a hierarchical object splitting, the step of dynamically selecting target text templates from a preset template library based on the image metadata, the task type identifier, and the anchor point position features includes: The intermediate image is globally parsed to obtain a list of object information; Based on the image metadata, the task type identifier, and the object information, multiple candidate text templates are selected from the preset template library; A comprehensive score is determined for each candidate text template based on static priority, historical success rate, and contextual matching degree. The candidate text templates are sorted according to the comprehensive score, and the main text template and backup text templates are determined based on the sorting results.

6. The method according to claim 5, characterized in that, The method further includes: The object name and canvas position are dynamically filled into the placeholders in the main text template to obtain the target main text template; By inputting the target main text template and the intermediate image into the target processing model, the object images of each object in the object information list are extracted from the intermediate image; The background of the object image is made transparent, and the object image with a transparent background is cropped to obtain the target object image.

7. The method according to claim 6, characterized in that, The method further includes at least one of the following: If the main text template fails to be invoked or the image quality of the target object image does not meet the standard, the backup text template will be automatically switched. If the backup text template fails to be invoked, the image processing task will be marked as pending review.

8. An image processing method, characterized in that, include: The first acquisition unit is used to acquire the task type identifier and the image resource identifier in response to the received image processing task; The reading unit is used to read the image metadata of the original image based on the image resource identifier; The preprocessing unit is used to preprocess the original image to obtain an intermediate image of the target size; and based on the task type identifier, to extract the spatial distribution information of each object in the intermediate image to form anchor point position features; The filtering unit is used to dynamically filter target text templates from a preset template library based on the image metadata, the task type identifier, and the anchor point position features. The first processing unit is used to input the target text template and the intermediate image into the target processing model to execute the image processing logic corresponding to the task type identifier and obtain the target processing result.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program can be executed by a terminal device or computer at runtime as described in any one of claims 1 to 7.

10. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to perform the method as described in any one of claims 1 to 7 via the computer program.

Images