Image processing method, system, device, electronic equipment, medium and program product

Abstract

Embodiments of the present application disclose an image processing method, system, device, electronic equipment, medium and program product. The method comprises: obtaining an initial image and processing instruction information corresponding to the initial image, and pre-processing the initial image based on the processing instruction information to generate initial feature data corresponding to the initial image and an image identifier; and sending the processing instruction information, the initial feature data and the image identifier to a server, so that the server performs image generation processing based on the processing instruction information, the initial feature data and the image identifier to obtain a target image corresponding to the initial image; and then receiving the target image returned by the server.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image processing method, system, apparatus, electronic device, medium, and program product. Background Technology

[0002] With the development of generative artificial intelligence technology, image generation services are increasingly being applied to mobile terminal scenarios, and users are placing higher demands on the real-time performance and stability of image-to-image generation. In existing technologies, image generation systems typically perform model inference operations on images through cloud servers. However, cloud servers need to allocate computing resources to perform image processing for a large number of requests, which can easily create a computing bottleneck. This leads to problems such as high latency and low resource utilization in the image processing process, making it difficult to meet the needs of real-time interaction. Summary of the Invention

[0003] This application provides an image processing method, system, apparatus, electronic device, medium, and program product, which can improve the real-time performance and stability of image generation. The above technical solutions are as follows: In a first aspect, embodiments of this application provide an image processing method applied to a user terminal, comprising: Obtain the initial image and the corresponding processing instruction information for the initial image; Based on the above processing instruction information, the above initial image is preprocessed to generate the initial feature data and image identifier corresponding to the above initial image; The processing instruction information, the initial feature data, and the image identifier are sent to the server so that the server performs image generation processing based on the processing instruction information, the initial feature data, and the image identifier to obtain the target image corresponding to the initial image. Receive the target image returned by the server.

[0004] In one possible implementation, the preprocessing of the initial image based on the processing instruction information to generate initial feature data and image identifiers corresponding to the initial image includes: Based on the above processing instruction information, a corresponding target processing strategy is determined. The target processing strategy includes a first processing strategy and / or a second processing strategy. The first processing strategy is used to perform structured feature extraction processing on the above initial image, and the second processing strategy is used to perform feature encoding processing on the above initial image. Based on the above target processing strategy and the above processing instruction information, the above initial image is preprocessed to generate the initial feature data and image identifier corresponding to the above initial image.

[0005] In one possible implementation, determining the corresponding target processing strategy based on the aforementioned processing instruction information includes: The above processing instruction information is semantically parsed to generate the corresponding processing task type; Determine whether the preset mapping relationship includes the preset processing strategy corresponding to the above-mentioned processing task type; If the preset mapping relationship includes the preset processing strategy corresponding to the above processing task type, the preset processing strategy corresponding to the above processing task type shall be determined as the target processing strategy. If the preset processing strategy corresponding to the above processing task type is not included in the above preset mapping relationship, the image attribute information of the above initial image is obtained, and the corresponding target processing strategy is determined based on the above image attribute information.

[0006] In one possible implementation, the above-mentioned target processing strategy determined based on the image attribute information includes: Based on the above image attribute information, the above initial image is subjected to image content evaluation processing to generate image structure evaluation parameters and image texture evaluation parameters corresponding to the above initial image. The above image structure evaluation parameters are used to characterize the completeness of spatial structure feature information in the above initial image, and the above image texture evaluation parameters are used to characterize the complexity of texture detail feature information in the above initial image. Based on the above image structure evaluation parameters, the above image texture evaluation parameters, and the preset evaluation parameter thresholds, the corresponding target processing strategy is determined.

[0007] In one possible implementation, the aforementioned preset evaluation parameter thresholds include a first preset threshold and a second preset threshold; determining the corresponding target processing strategy based on the aforementioned image structure evaluation parameters, the aforementioned image texture evaluation parameters, and the preset evaluation parameter thresholds includes: If the above image structure evaluation parameter is greater than the first preset threshold and the above image texture evaluation parameter is less than the second preset threshold, the above first processing strategy shall be used as the corresponding target processing strategy. If the above image structure evaluation parameter is not greater than the above first preset threshold and the above image texture evaluation parameter is greater than the above second preset threshold, the above second processing strategy shall be used as the corresponding target processing strategy. If the above image structure evaluation parameter is greater than the above first preset threshold and the above image texture evaluation parameter is not less than the above second preset threshold, the above first processing strategy and the above second processing strategy are used as the corresponding target processing strategy. If the image structure evaluation parameters are not greater than the first preset threshold and the image texture evaluation parameters are not greater than the second preset threshold, the second processing strategy will be used as the corresponding target processing strategy.

[0008] In one possible implementation, when the target processing strategy includes the first processing strategy and the second processing strategy, the preprocessing of the initial image based on the target processing strategy and the processing instruction information to generate initial feature data and image identifiers corresponding to the initial image includes: The initial image above is subjected to structured feature extraction processing to generate structured feature data; The initial image is subjected to feature encoding processing to generate latent spatial feature data; The structured feature data and the potential spatial feature data are fused together to generate fused feature data. The aforementioned fused feature data is determined as the initial feature data corresponding to the aforementioned initial image, and an image identifier corresponding to the aforementioned initial feature data is generated.

[0009] In one possible implementation, the acquisition of the initial image and the corresponding processing instruction information includes: Obtain the initial image determined by the user, and obtain the task description information determined by the user for the initial image; The above task description information is semantically parsed to generate corresponding task semantic feature information; Based on the above task semantic feature information and the preset task template library, a matching analysis is performed to determine the target processing task type and the corresponding set of processing parameters. The above processing instruction information is generated based on the above target processing task type and the above processing parameter set.

[0010] Secondly, embodiments of this application provide an image processing method applied to a server, comprising: The system receives processing instruction information, initial feature data, and image identifiers sent by a user terminal, wherein the initial feature data and the image identifiers are data generated by the user terminal through preprocessing of the initial image based on the processing instruction information. Based on a preset diffusion model, the above processing instruction information, the above initial feature data and the above image identifier are processed to generate an image, thereby obtaining the target image corresponding to the above initial image. The target image is returned to the user terminal.

[0011] In one possible implementation, the above-mentioned image generation processing based on a preset diffusion model of the processing instruction information, the initial feature data, and the image identifier to obtain the target image corresponding to the initial image includes: The above processing instruction information, the above initial feature data and the above image identifier are input into the above preset diffusion model to perform denoising processing at multiple time steps; For each of the above multiple time steps, obtain the current intermediate feature data corresponding to the current time step, and calculate the feature similarity between the current intermediate feature data and the historical intermediate feature data of the previous time step; If the similarity of the above features is greater than the preset similarity threshold, the above denoising processing in the current time step is performed based on the above historical intermediate feature data; If the similarity of the above features is not greater than the above preset similarity threshold, denoising is performed based on the above processing instruction information and the above current intermediate feature data to generate new intermediate feature data; Under the condition that the preset iteration stopping condition is met, the target image corresponding to the initial image is generated by decoding the new intermediate feature data through the preset diffusion model.

[0012] Thirdly, embodiments of this application provide an image processing system, including a user terminal and a server; wherein the user terminal is used to execute the method provided in the first aspect, and the server is used to execute the method provided in the second aspect.

[0013] Fourthly, embodiments of this application provide an image processing apparatus applied to a user terminal, comprising: The first acquisition module is used to acquire the initial image and the processing instruction information corresponding to the initial image; The first processing module is used to preprocess the initial image based on the processing instruction information to generate initial feature data and image identifiers corresponding to the initial image. The first sending module is used to send the above-mentioned processing instruction information, the above-mentioned initial feature data and the above-mentioned image identifier to the server, so that the server performs image generation processing based on the above-mentioned processing instruction information, the above-mentioned initial feature data and the above-mentioned image identifier to obtain the target image corresponding to the above-mentioned initial image; The first receiving module is used to receive the target image returned by the server.

[0014] Fifthly, embodiments of this application provide an image processing apparatus applied to a server, comprising: The second receiving module is used to receive processing instruction information, initial feature data and image identifier sent by the user terminal, wherein the initial feature data and the image identifier are data generated by the user terminal by preprocessing the initial image based on the processing instruction information. The second processing module is used to perform image generation processing on the above-mentioned processing instruction information, the above-mentioned initial feature data and the above-mentioned image identifier based on a preset diffusion model, so as to obtain the target image corresponding to the above-mentioned initial image. The second sending module is used to return the target image to the user terminal.

[0015] Sixthly, embodiments of this application provide an electronic device, including: a processor and a memory; wherein the memory stores a computer program, and the processor executes the computer program to implement the method steps provided in the first aspect of embodiments of this application, or the processor executes the computer program to implement the method steps provided in the second aspect of embodiments of this application.

[0016] In a seventh aspect, embodiments of this application provide a computer storage medium storing a plurality of instructions, which are adapted to be loaded by a processor and executed by the method steps provided in the first aspect of embodiments of this application, or the instructions are adapted to be loaded by a processor and executed by the method steps provided in the second aspect of embodiments of this application.

[0017] Eighthly, embodiments of this application provide a computer program product containing instructions that, when run on a computer or processor, cause the computer or processor to execute the image processing method provided in the first or second aspect of embodiments of this application.

[0018] This application embodiment obtains an initial image and corresponding processing instruction information; preprocesses the initial image based on the processing instruction information to generate initial feature data and image identifiers corresponding to the initial image; and sends the processing instruction information, initial feature data, and image identifiers to a server, so that the server performs image generation processing based on the processing instruction information, initial feature data, and image identifiers to obtain a target image corresponding to the initial image; and then receives the target image returned by the server. Thus, by moving the image preprocessing operation to the user terminal side and scheduling the preprocessing strategy in conjunction with the processing instruction information, the image generation processing flow under the edge-cloud collaborative computing architecture is optimized. On the one hand, it effectively reduces the computing load and memory usage of the server in high-concurrency scenarios, improving the efficiency of cloud resource utilization; on the other hand, by generating initial feature data and image identifiers on the terminal side, it reduces the amount of original image data transmitted, shortens network transmission latency, and thus effectively improves the real-time performance and stability of image generation. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1An exemplary system architecture diagram of an image processing method provided in this application embodiment; Figure 2 A schematic flowchart of an image processing method provided in an embodiment of this application; Figure 3 A flowchart illustrating a method for generating initial feature data and image identifiers provided in an embodiment of this application; Figure 4 A flowchart illustrating a target processing strategy determination method provided in an embodiment of this application; Figure 5 A flowchart illustrating another initial feature data and image identification method provided in an embodiment of this application; Figure 6 A flowchart illustrating another image processing method provided in an embodiment of this application; Figure 7 A schematic flowchart illustrating an image generation method provided in an embodiment of this application; Figure 8 This is a schematic flowchart illustrating a specific image processing method provided in an embodiment of this application; Figure 9 This is a schematic diagram of the structure of a first image processing apparatus provided in an embodiment of this application; Figure 10 This is a schematic diagram of the structure of a second image processing apparatus provided in an embodiment of this application; Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0021] To make the features and advantages of the embodiments of this application more apparent and understandable, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the embodiments of this application.

[0022] In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims. Furthermore, in the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; the word "and / or" in the text is merely a description of the association relationship between related objects, indicating that three relationships can exist, for example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Additionally, in the description of the embodiments of this application, "multiple" refers to two or more.

[0023] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0024] With the development of generative artificial intelligence technology, image generation services are increasingly being applied to mobile terminal scenarios, and users are placing higher demands on the real-time performance and stability of image-to-image generation. In existing technologies, image generation systems typically perform model inference operations on images through cloud servers. However, cloud servers need to allocate computing resources to perform image processing for a large number of requests, which can easily create a computing bottleneck. This leads to problems such as high latency and low resource utilization in the image processing process, making it difficult to meet the needs of real-time interaction.

[0025] Therefore, this application provides an image processing method to solve the technical problems of low real-time performance and low stability in image generation.

[0026] Please see Figure 1 , Figure 1 This is an exemplary system architecture diagram of an image processing method provided in an embodiment of this application.

[0027] like Figure 1 As shown, the system architecture may include a user terminal 101, a network 102, and a server 103. The network 102 serves as the medium for providing a communication link between the user terminal 101 and the server 103. The network 102 may include various types of wired or wireless communication links, such as wired communication links including fiber optic cables, twisted-pair cables, or coaxial cables, and wireless communication links including Bluetooth communication links, Wireless-Fidelity (Wi-Fi) communication links, or microwave communication links, etc.

[0028] User terminal 101 can interact with server 103 via network 102 to receive messages from or send messages to server 103. Alternatively, user terminal 101 can interact with server 103 via network 102 to receive messages or data sent to server 103 by other users. User terminal 101 can be hardware or software. When user terminal 101 is hardware, it can be various electronic devices, including but not limited to tablet computers, laptops, and desktop computers. When user terminal 101 is software, it can be installed in the electronic devices listed above, and it can be implemented as multiple software programs or software modules (e.g., to provide distributed services) or as a single software program or software module; no specific limitation is made here.

[0029] In this embodiment, the user terminal 101 first obtains an initial image and processing instruction information corresponding to the initial image; then preprocesses the initial image based on the processing instruction information to generate initial feature data and image identifier corresponding to the initial image; and sends the processing instruction information, the initial feature data, and the image identifier to the server 103, so that the server 103 performs image generation processing based on the processing instruction information, the initial feature data, and the image identifier to obtain a target image corresponding to the initial image; and then receives the target image returned by the server 103.

[0030] Server 103 can be a server that provides various services. It should be noted that server 103 can be hardware or software. When server 103 is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When server 103 is software, it can be implemented as multiple software programs or software modules (e.g., used to provide distributed services), or as a single software program or software module; no specific limitations are made here.

[0031] It should be understood that Figure 1 The number of user terminals, networks, and servers shown is only illustrative; the number can be any number of user terminals, networks, and servers depending on the implementation requirements.

[0032] Please see Figure 2 , Figure 2 This is a schematic flowchart illustrating an image processing method provided in an embodiment of this application. Figure 2 As shown, an image processing method may include at least: S202: Obtain the initial image and the processing instruction information corresponding to the initial image.

[0033] The initial image can refer to the image data to be processed obtained by the user terminal through an image acquisition device, image storage module, or network download module. The initial image can be a photo taken by the user, a screenshot of the screen, a historical image file, or an image resource obtained from a third-party platform.

[0034] Optionally, the processing instruction information can be a set of control information that characterizes the user's image generation requirements. Specifically, it may include processing task type, style parameters, resolution parameters, generation intensity parameters, and random seed parameters, etc., to guide the execution method of subsequent image generation processes.

[0035] Specifically, the user terminal can obtain the initial image selected or uploaded by the user through the user interface and receive the task description information input by the user. This task description information can be obtained through text input, voice input, or selection of a preset template. The user terminal performs semantic parsing on the task description information, generates corresponding processing instructions, and stores them in association with the initial image.

[0036] S204: Based on the above processing instruction information, preprocess the above initial image to generate initial feature data and image identifiers corresponding to the above initial image.

[0037] Preprocessing can refer to the process of performing at least one operation on the initial image, such as structural feature extraction, feature encoding, normalization, and format conversion. The aforementioned initial feature data can be the feature representation data of the initial image, used to characterize information such as the structured feature data and latent spatial feature data of the initial image.

[0038] Optionally, the aforementioned image identifier can be initial feature data or unique identifier information corresponding to the initial image, used to achieve request matching, cache management and result reuse during the end-to-cloud collaboration process.

[0039] Specifically, the user terminal can determine the corresponding target processing strategy based on the processing instruction information, and perform structured feature extraction and / or feature encoding processing on the initial image to obtain initial feature data. For example, when the processing instruction information indicates a relatively detailed redrawing task, the user terminal can extract edge features and depth features from the initial image; when the processing instruction information indicates a rapid generation task, the user terminal can perform latent spatial encoding processing on the initial image. Furthermore, the user terminal can generate corresponding image identifiers based on the initial feature data. For example, the aforementioned image identifiers may include identifier information generated based on hash operations.

[0040] S206: The above-mentioned processing instruction information, the above-mentioned initial feature data and the above-mentioned image identifier are sent to the server, so that the server performs image generation processing based on the above-mentioned processing instruction information, the above-mentioned initial feature data and the above-mentioned image identifier to obtain the target image corresponding to the above-mentioned initial image.

[0041] The server can refer to a server cluster or cloud computing platform that has deployed an image generation model (such as a diffusion model) to receive data uploaded by user terminals and perform diffusion model inference calculations.

[0042] Optionally, the target image mentioned above can be the final image result generated by the server through a diffusion model, which is presented to the user as a processing output.

[0043] Specifically, the user terminal can package processing instruction information, initial feature data, and image identifiers into image generation request data, and send it to the server through the relevant network communication module. During the transmission process, the user terminal can perform compression encoding and fragmented transmission processing on the initial feature data to reduce network bandwidth consumption and improve transmission stability.

[0044] S208: Receive the target image returned by the server.

[0045] Specifically, the user terminal can receive the target image returned by the server via the network, decode and render the target image, and display the target image in the user interface. Furthermore, the user terminal can associate and store the target image with corresponding processing instruction information, allowing the user to perform secondary editing or historical rewinding operations.

[0046] This application embodiment obtains an initial image and corresponding processing instruction information; preprocesses the initial image based on the processing instruction information to generate initial feature data and image identifiers corresponding to the initial image; and sends the processing instruction information, initial feature data, and image identifiers to a server, so that the server performs image generation processing based on the processing instruction information, initial feature data, and image identifiers to obtain a target image corresponding to the initial image; and then receives the target image returned by the server. Thus, by moving the image preprocessing operation to the user terminal side and scheduling the preprocessing strategy in conjunction with the processing instruction information, the image generation processing flow under the edge-cloud collaborative computing architecture is optimized. On the one hand, it effectively reduces the computing load and memory usage of the server in high-concurrency scenarios, improving the efficiency of cloud resource utilization; on the other hand, by generating initial feature data and image identifiers on the terminal side, it reduces the amount of original image data transmitted, shortens network transmission latency, and thus effectively improves the real-time performance and stability of image generation.

[0047] In one embodiment, in S202 above, obtaining the initial image and the processing instruction information corresponding to the initial image may include the following steps: obtaining the initial image determined by the user, and obtaining the task description information determined by the user for the initial image; performing semantic parsing processing on the task description information to generate corresponding task semantic feature information; performing matching analysis based on the task semantic feature information and a preset task template library to determine the target processing task type and the corresponding processing parameter set; and generating the processing instruction information according to the target processing task type and the processing parameter set.

[0048] The task description information mentioned above can be input information used to characterize the user's image processing needs. Specifically, it can include user-inputted text description information, speech-to-text information, preset option combination information, or interactive editing information, which are used to reflect the user's requirements for the generation method, style characteristics, and processing target of the target image.

[0049] Optionally, the aforementioned task semantic feature information can be used to characterize the semantic content, intent category, and parameter indication information contained in the task description information.

[0050] Optionally, the aforementioned preset task template library can be a collection of task templates pre-built and stored in the user terminal or server. This collection contains multiple task templates, each of which can describe a typical image generation task type and its corresponding parameter configuration rules. The aforementioned task template library can be built through historical user behavior analysis, manual configuration, or model training, and can be dynamically updated according to actual application needs.

[0051] It should be noted that the above-mentioned target processing task types can be used to characterize the execution mode and processing objectives of the current image generation task. For example, target processing task types may include fine redrawing tasks, style transfer tasks, rapid generation tasks, image enhancement tasks, etc.

[0052] In this embodiment, the processing parameter set can be a parameter configuration data set corresponding to the target processing task type, used to control the model inference method and output effect during the image generation process. For example, the processing parameter set may include generation step parameters, resolution parameters, style intensity parameters, random seed parameters, noise control parameters, etc.

[0053] In one specific embodiment, the user terminal first obtains an initial image selected by the user through a user interface and receives task description information input by the user for the initial image. This task description information can be text content input by the user, for example, "convert this image to a watercolor style while maintaining clear character outlines"; it can also be text information converted through speech recognition, or a combination of instructions formed based on selections within a graphical interface. Subsequently, the user terminal performs semantic parsing processing on the obtained task description information, such as using a natural language processing model to perform word segmentation analysis, semantic encoding, and intent recognition processing on the text content, generating corresponding task semantic feature information. Then, the user terminal performs matching analysis between the task semantic feature information and various task templates in a preset task template library, determines the target processing task type corresponding to the current task semantic feature information based on the matching results, and extracts a set of processing parameters associated with the target processing task type. Further, the user terminal generates corresponding processing instruction information based on the target processing task type and the set of processing parameters, and stores the processing instruction information in association with the initial image for subsequent image preprocessing and image generation operations.

[0054] In this embodiment, by semantically parsing the user-input task description information and performing matching analysis with a preset task template library, the target processing task type and corresponding set of processing parameters are automatically determined, achieving a precise understanding and structured expression of the user's image processing needs. On the one hand, this avoids the operational burden of manually configuring complex generation parameters, improving the automation level of image generation task configuration; on the other hand, through a standardized task template and parameter mapping mechanism, the stability and consistency of the processing instruction generation process are improved, reducing the risk of poor generation results due to unreasonable parameter settings.

[0055] In one embodiment, such as Figure 3 In S204 above, the preprocessing of the initial image based on the processing instruction information to generate initial feature data and image identifiers corresponding to the initial image may include the following steps: S302: Based on the above processing instruction information, determine the corresponding target processing strategy, which includes a first processing strategy and / or a second processing strategy. The first processing strategy is used to perform structured feature extraction processing on the initial image, and the second processing strategy is used to perform feature encoding processing on the initial image.

[0056] The aforementioned target processing strategy can be a preprocessing execution scheme, used to indicate the specific feature processing methods and processes to be applied to the current initial image. Optionally, the aforementioned target processing strategy can include a single processing strategy or a combination of multiple processing strategies to adapt to the processing requirements of different image generation tasks.

[0057] Optionally, the first processing strategy described above can be a preprocessing strategy for performing structured feature extraction on the initial image. Specifically, it can be used to extract spatial structural features such as contour information, edge information, spatial layout information, and target object structural information from the initial image. The structured feature data generated by the first processing strategy can be used to enhance the ability of the cloud-based diffusion model to constrain the overall structure of the image.

[0058] Optionally, the second processing strategy described above can be a preprocessing strategy for performing feature encoding on the initial image. Specifically, it can be used to map the initial image to a low-dimensional latent space feature representation to generate corresponding latent space feature data. This second processing strategy can effectively reduce the transmission scale and computational complexity of image data, and improve the efficiency of edge-cloud collaborative processing.

[0059] S304: Based on the above target processing strategy and the above processing instruction information, the above initial image is preprocessed to generate the initial feature data and image identifier corresponding to the above initial image.

[0060] In one specific embodiment, after the user terminal obtains processing instruction information and determines the target processing strategy, it performs corresponding preprocessing operations on the initial image according to the target processing strategy. Specifically, when the target processing strategy is a first processing strategy, the user terminal performs structured feature extraction processing on the initial image to generate structured feature data; when the target processing strategy is a second processing strategy, the user terminal performs feature encoding processing on the initial image to generate latent spatial feature data; when the target processing strategy is a combination of the first and second processing strategies, the user terminal performs structured feature extraction processing and feature encoding processing respectively, and performs fusion processing on the generated feature data to form initial feature data. Further, the user terminal generates a corresponding image identifier based on the initial feature data, and associates and stores the initial feature data and image identifier with the processing instruction information for subsequent edge-cloud collaborative image generation processing.

[0061] This application embodiment improves the ability of initial feature data to represent the spatial structure information and texture details of the original image by performing targeted structured feature extraction or feature encoding processing, and enhances the constraint effect of the cloud diffusion model on the generated image. In addition, by flexibly switching and fusing multiple preprocessing paths, unnecessary repetitive calculations and data transmission overhead are reduced, and the utilization efficiency of edge computing resources and overall processing performance are improved.

[0062] In one embodiment, such as Figure 4 In step S302 above, determining the corresponding target processing strategy based on the processing instruction information may include the following steps: S402: Perform semantic parsing on the above processing instruction information to generate the corresponding processing task type.

[0063] The aforementioned processing task types can be used to characterize the main processing objectives and execution modes of the current image generation task. For example, these processing task types may include, but are not limited to, detailed redrawing tasks, style transfer tasks, rapid generation tasks, structure preservation tasks, and image enhancement tasks.

[0064] Specifically, the user terminal can perform semantic parsing on the acquired processing instruction information. For example, it can use natural language processing models or rule matching methods to perform word segmentation analysis, semantic encoding, and intent recognition on the processing instruction information, thereby generating the corresponding processing task type. For instance, when the processing instruction information contains words such as "preserve outline" and "fine redraw," the user terminal can determine the corresponding processing task type as a structure-preserving task.

[0065] For example, a user terminal can have a built-in or invoked natural language processing (NLP) model to perform semantic parsing of processing instruction information. This NLP model can be a text understanding model built on deep learning, with the user-inputted task description text as input and the corresponding processing task type as output. Specifically, after obtaining the processing instruction information, the user terminal first performs word segmentation and vectorization encoding on the text content, converting the processing instruction information into a corresponding semantic feature representation. Then, this semantic feature representation is input into the NLP model for inference calculation, obtaining the corresponding semantic encoding result. The user terminal further performs intent recognition and task classification processing based on the semantic encoding result to determine and output the processing task type corresponding to the current processing instruction information.

[0066] It is understood that the aforementioned natural language processing model can be trained based on a pre-constructed training sample dataset. This training sample dataset can include multiple sample pairs, each pair comprising a task description text sample and a corresponding target task type label. For example, the training sample dataset can include data samples in the following forms: converting an image to an oil painting style – style transfer task; redrawing a person's outline while preserving their features – structure preservation task; quickly generating cartoon-style images – quick generation task. During model training, the task description text samples are first preprocessed, including word segmentation, denoising, and semantic encoding, to generate corresponding text feature representations. Subsequently, these text feature representations are input into the natural language processing model for forward computation to obtain the predicted task type result. This prediction result is then compared with the corresponding target task type label, and a classification loss function is calculated. Further, the model parameters are updated via backpropagation based on the loss function to continuously optimize the natural language processing model's understanding of the semantic features of the task description. Through multiple rounds of iterative training, the natural language processing model gradually converges and can accurately identify the processing task type corresponding to different task description information. Once trained, the natural language processing model can be deployed locally on the user's terminal and continuously updated and optimized based on actual running data.

[0067] S404: Determine whether the preset mapping relationship includes the preset processing strategy corresponding to the above-mentioned processing task type.

[0068] The aforementioned preset mapping relationship can be a pre-established correspondence table between processing task types and preset processing strategies, used to describe the mapping rules between different processing task types and their corresponding preset processing strategies. The aforementioned preset processing strategy can be a standardized preprocessing execution scheme, used to indicate the default feature processing method and processing flow to be adopted for tasks of a corresponding processing task type. Optionally, the aforementioned preset processing strategy may include a first processing strategy, a second processing strategy, or a combination of the first and second processing strategies.

[0069] It is understandable that the above-mentioned preset mapping relationship can be stored locally on the user terminal or on the server, and can be updated and maintained according to the system operation.

[0070] S406: If the preset mapping relationship includes the preset processing strategy corresponding to the above processing task type, the preset processing strategy corresponding to the above processing task type shall be determined as the target processing strategy.

[0071] Specifically, the user terminal can perform a search and matching operation in a preset mapping relationship based on the determined processing task type to determine whether a preset processing strategy corresponding to the current processing task type exists. For example, the user terminal can query the processing strategy configuration rules corresponding to the fine redraw task in the mapping relationship table. Furthermore, when the user terminal detects that the preset mapping relationship includes a preset processing strategy corresponding to the current processing task type, it can directly determine the aforementioned preset processing strategy as the target processing strategy.

[0072] For example, in the preset mapping relationship, there is a quick generation task (processing task type), and the corresponding preset processing strategy is the second processing strategy. Then, when the current processing task type is a quick generation task, the user terminal can directly select the second processing strategy corresponding to this processing task type as the target processing strategy.

[0073] S408: If the preset processing strategy corresponding to the above processing task type is not included in the above preset mapping relationship, obtain the image attribute information of the above initial image, and determine the corresponding target processing strategy based on the above image attribute information.

[0074] Image attribute information refers to descriptive feature information obtained after analyzing the initial image, which reflects the content and quality characteristics of the initial image. Optionally, the aforementioned image attribute information may include image resolution information, sharpness information, structural complexity information, texture complexity information, brightness distribution information, noise level information, and subject proportion information, etc.

[0075] It is understandable that the preset mapping relationship does not include the preset processing strategies corresponding to the above-mentioned processing task types. This may indicate that the current processing task type is not within the scope of the system's predefined standard task templates, or that the current task description information is combinable, ambiguous, or complex, making it difficult to directly determine the corresponding processing strategy through fixed mapping rules. In this case, adaptive strategy determination can be performed, dynamically determining the corresponding target processing strategy by analyzing the image attribute information of the initial image.

[0076] In one specific embodiment, the user terminal can build upon or invoke an image attribute analysis model to analyze and process the initial image to obtain corresponding image attribute information. This image attribute analysis model can be an image feature extraction model built based on deep learning. The model input is the initial image, and the output is a multi-dimensional attribute parameter vector representing the image content and quality features. Specifically, after acquiring the initial image, the user terminal can first perform operations such as size normalization, grayscale normalization, and noise suppression on the initial image, and then input the processed image into the image attribute analysis model for inference calculations. The image attribute analysis model performs multi-layer feature extraction and feature fusion processing on the input image to generate corresponding image feature representations, and further outputs image attribute information such as image sharpness parameters, structural complexity parameters, texture complexity parameters, brightness distribution parameters, and noise level parameters based on these image feature representations. The user terminal can normalize and weight the various attribute parameters output by the model to form a unified set of image attribute information, i.e., image attribute information.

[0077] It is understood that the aforementioned image attribute analysis model can be trained based on a pre-constructed training sample dataset. This training sample dataset may include multiple image samples and corresponding attribute annotations for each image sample. The attribute annotations characterize the image sample's sharpness level, structural complexity level, texture complexity level, noise level, and subject proportion level. For example, the training sample dataset may include sample data in the following forms: high-resolution sharp images – high structural complexity, high texture complexity; blurry low-resolution images – low sharpness, high noise level; simple background images – low structural complexity, low texture complexity. During model training, the training sample images are first subjected to data augmentation processing, including rotation, cropping, scaling, and brightness perturbation, to improve the model's generalization ability. Subsequently, the processed image samples are input into the image attribute analysis model for forward computation to obtain predicted attribute parameters. These predicted results are then compared with the corresponding attribute annotations to calculate the attribute regression loss function or classification loss function. Furthermore, based on the aforementioned loss function, the model parameters are updated and optimized using the backpropagation algorithm, enabling the image attribute analysis model to gradually learn the mapping relationship between image features and attribute information. Through multiple rounds of iterative training, the image attribute analysis model converges and can stably output image attribute information that conforms to the actual image features. The trained image attribute analysis model can be deployed locally on the user terminal and continuously updated and optimized based on actual operating data.

[0078] Optionally, if the user terminal does not find a preset processing strategy corresponding to the current processing task type in the preset mapping relationship, the user terminal can further analyze and process the initial image to obtain the corresponding image attribute information. For example, the user terminal can perform sharpness detection, structural complexity assessment, and texture density analysis on the initial image to generate corresponding image attribute information. Subsequently, the user terminal can determine the target processing strategy based on the above image attribute information. For example, when the image structure is highly complex and rich in texture details, the user terminal prioritizes the first processing strategy; when the image structure is simple and the resolution is low, the user terminal prioritizes the second processing strategy.

[0079] In this embodiment, the accuracy and efficiency of the target processing strategy determination process are improved by combining semantic understanding with rule mapping. Furthermore, by introducing an image attribute analysis model as a fallback decision-making method, the failure of strategy selection due to instruction ambiguity or rule missing is effectively reduced, and the adaptability to complex scenarios and abnormal inputs is enhanced.

[0080] In one embodiment, in step 408 above, determining the corresponding target processing strategy based on the image attribute information may include the following steps: performing image content evaluation processing on the initial image based on the image attribute information to generate image structure evaluation parameters and image texture evaluation parameters corresponding to the initial image, wherein the image structure evaluation parameters are used to characterize the completeness of spatial structure feature information in the initial image, and the image texture evaluation parameters are used to characterize the complexity of texture detail feature information in the initial image; and determining the corresponding target processing strategy based on the image structure evaluation parameters, the image texture evaluation parameters, and a preset evaluation parameter threshold.

[0081] Optionally, the aforementioned image structure evaluation parameters can reflect structural attributes such as the sharpness of the subject contour, edge continuity, spatial layout consistency, and target region integrity in the initial image. For example, the image structure evaluation parameters can be calculated comprehensively based on parameters such as edge intensity distribution, contour connectivity index, target segmentation confidence, and depth consistency index.

[0082] Optionally, the aforementioned image texture evaluation parameters can reflect texture attributes such as pixel detail distribution density, texture direction variation amplitude, gray-level co-occurrence relationship characteristics, and local frequency domain energy distribution characteristics in the initial image. For example, the image texture evaluation parameters can be obtained through comprehensive calculation based on local binary mode features, gray-level co-occurrence matrix features, and frequency domain spectral energy characteristics.

[0083] It should be noted that the larger the value of the image structure evaluation parameter, the more complete the spatial structure feature information in the represented initial image. The larger the value of the image texture evaluation parameter, the more complex the texture detail feature information in the represented initial image.

[0084] In one specific embodiment, the user terminal can input the initial image into a pre-trained image content evaluation model. The image content evaluation model performs feature extraction and comprehensive analysis on the initial image, automatically generating corresponding image structure evaluation parameters and image texture evaluation parameters. The image content evaluation model can be an image analysis model built based on deep learning, which can achieve a quantitative evaluation of the integrity of image structure and the complexity of texture by jointly modeling the spatial structure feature information and texture detail feature information in the initial image.

[0085] It is understood that the aforementioned image content evaluation model can be trained based on a pre-constructed training sample dataset. This training sample dataset includes multiple image samples and corresponding image attribute annotations. The image attribute annotations characterize the structural integrity and texture complexity of the image samples. For example, the image attribute annotations may include labels such as structural integrity level, texture complexity level, sharpness level, and noise level, and can be obtained through manual annotation, automatic evaluation tool generation, or a combination of both. During model training, the training sample images are first preprocessed, including size normalization, color normalization, and noise suppression, to improve the stability of model training. Subsequently, the preprocessed image samples are input into the image content evaluation model for forward computation to obtain the corresponding predicted values ​​for structural evaluation parameters and texture evaluation parameters. Further, the predicted values ​​are compared with the corresponding image attribute annotations to calculate the evaluation error loss function. Based on this loss function, the model parameters are updated and optimized using a backpropagation algorithm, enabling the image content evaluation model to gradually learn the mapping relationship between image features and image attributes. Through multiple rounds of iterative training, the image content evaluation model gradually converges and can stably output image structure evaluation parameters and image texture evaluation parameters that conform to the actual image characteristics. The trained image content evaluation model can be deployed locally on the user terminal and continuously updated and optimized based on actual operating data.

[0086] In one embodiment, the aforementioned preset evaluation parameter threshold may include a first preset threshold and a second preset threshold; wherein, the first preset threshold may be used to classify and determine the image structure evaluation parameters to distinguish whether the spatial structure feature information in the initial image is in a high integrity state or a low integrity state; the second preset threshold may be used to classify and determine the image texture evaluation parameters to distinguish whether the texture detail feature information in the initial image is in a high complexity state or a low complexity state.

[0087] Specifically, when the image structure evaluation parameter is greater than a first preset threshold, it indicates that the current initial image has high structural integrity and contour clarity; when the image structure evaluation parameter is less than or equal to the first preset threshold, it indicates that the structural features of the current initial image are relatively blurry or missing. When the image texture evaluation parameter is greater than or equal to a second preset threshold, it indicates that the current initial image has high texture detail complexity; when the image texture evaluation parameter is less than the second preset threshold, it indicates that the texture detail information of the current initial image is relatively simple.

[0088] In one specific embodiment, the first preset threshold and the second preset threshold can be determined based on the statistical analysis results of historical image sample data. For example, the distribution statistical analysis of the structural evaluation parameters and texture evaluation parameters corresponding to a large number of historical image samples can be performed in advance, and the corresponding reference threshold range can be determined based on the preset confidence interval, quantile interval or cluster analysis results, and the first preset threshold and the second preset threshold can be selected from the above reference threshold range.

[0089] Optionally, the first and second preset thresholds can also be adaptively adjusted based on the training results of the image content evaluation model. Specifically, during model training, the generation effect quality indicators corresponding to different parameter ranges can be statistically analyzed simultaneously, and the first and second preset thresholds can be dynamically optimized based on the generation quality feedback results, so that the threshold settings better meet the needs of actual application scenarios.

[0090] In this embodiment, by using an image content evaluation model to automatically analyze and process the initial image, the spatial structure feature information and texture detail feature information in the initial image can be jointly modeled and quantitatively evaluated, generating corresponding image structure evaluation parameters and image texture evaluation parameters. This achieves an objective characterization of the integrity of the image structure and the complexity of the texture, effectively improving the accuracy and stability of image feature evaluation.

[0091] Optionally, determining the corresponding target processing strategy based on the image structure evaluation parameters, the image texture evaluation parameters, and the preset evaluation parameter thresholds includes: when the image structure evaluation parameters are greater than a first preset threshold and the image texture evaluation parameters are less than a second preset threshold, using the first processing strategy as the corresponding target processing strategy; when the image structure evaluation parameters are not greater than the first preset threshold and the image texture evaluation parameters are greater than the second preset threshold, using the second processing strategy as the corresponding target processing strategy; when the image structure evaluation parameters are greater than the first preset threshold and the image texture evaluation parameters are not less than the second preset threshold, using both the first and second processing strategies as the corresponding target processing strategies; and when the image structure evaluation parameters are not greater than the first preset threshold and the image texture evaluation parameters are not greater than the second preset threshold, using the second processing strategy as the corresponding target processing strategy.

[0092] Specifically, when the image structure evaluation parameter is greater than a first preset threshold and the image texture evaluation parameter is less than a second preset threshold, it indicates that the current initial image has relatively clear spatial structure features but relatively simple texture details. In this case, the first processing strategy can be prioritized to enhance the ability of the generated image to preserve the original structural information. When the image structure evaluation parameter is not greater than (i.e., less than or equal to) the first preset threshold and the image texture evaluation parameter is not less than (i.e., greater than or equal to) the second preset threshold, it indicates that the current initial image has relatively weak structural information but rich texture details. In this case, the user terminal prioritizes the second processing strategy to improve the encoding efficiency of detailed features. When the image structure evaluation parameter is greater than the first preset threshold and the image texture evaluation parameter is not less than the second preset threshold, it indicates that the current initial image simultaneously possesses high structural integrity and texture complexity. The first and second processing strategies can be combined to determine the target processing strategy to balance structural constraints and detail expression capabilities. When the image structure evaluation parameter is not greater than the first preset threshold and the image texture evaluation parameter is not greater than the second preset threshold, it indicates that the overall information content of the current initial image is low. In this case, the second processing strategy can be prioritized to reduce computational complexity and improve overall processing efficiency.

[0093] In this embodiment, by collaboratively analyzing image structural and texture information, the degradation in generation quality caused by the mismatch between preprocessing strategies and actual image features is effectively reduced. Furthermore, while improving image generation quality, the allocation of computational resources can be dynamically adjusted based on the level of image information. When structural information is strong, structural constraints are strengthened; when texture information is rich, detail encoding efficiency is improved; when comprehensive features are high, multi-strategy collaborative processing is implemented; and when overall information content is low, unnecessary computational overhead is reduced, thus achieving a balanced optimization between generation quality and processing efficiency. Therefore, this embodiment not only improves the accuracy and flexibility of edge-side preprocessing strategy selection but also enhances the adaptability of the edge-cloud collaborative image generation system to diverse image scenes, significantly improving overall stability, operational efficiency, and user experience.

[0094] In one embodiment, such as Figure 5 When the target processing strategy includes the first processing strategy and the second processing strategy, in step S304, preprocessing the initial image based on the target processing strategy and the processing instruction information to generate initial feature data and image identifiers corresponding to the initial image may include the following steps: S502: Perform structured feature extraction processing on the above initial image to generate structured feature data.

[0095] The structured feature data mentioned above can be used to characterize the spatial structural features of the target object in the initial image. Specifically, the structured feature data may include contour features, edge distribution features, spatial layout features, target region segmentation information, and depth structure features in the initial image.

[0096] For example, the user terminal can integrate a lightweight image processing library locally, such as a cross-platform Open Source Computer Vision Library (OpenCV.js) module or a shader-based image filter processing module. Specifically, the terminal device can generate a corresponding edge feature map (CannyMap) based on the Canny algorithm to extract edge intensity distribution information in the image; it can generate a line art feature map (Lineart) based on line art extraction algorithms or contour detection algorithms to obtain the contour continuity and structural layout information of the target object; and it can generate a depth map (Depth Map) based on a lightweight depth estimation model or gray-level gradient mapping algorithm to represent the spatial hierarchy and foreground / background structure information in the image. The aforementioned feature maps, such as edge maps, line art maps, and depth maps, can be further processed with unified size adjustment, channel alignment, and numerical normalization to form structured feature data in a standard format.

[0097] It should be noted that the above-described structured feature extraction process is fully automated. The generated edge maps, line drawings, or depth maps are not displayed in the user interface, but are directly packaged with the processing instruction information as conditional control features and sent to the server as input conditions for the diffusion model. By completing the structural feature calculation on the device side in advance, it is possible to avoid loading a large-scale preprocessing model on the server side, thereby reducing cloud memory usage and model initialization time, improving the overall image generation efficiency, and reducing the computational pressure in high-concurrency scenarios.

[0098] S504: Perform feature encoding processing on the above initial image to generate latent spatial feature data.

[0099] The aforementioned latent spatial feature data can be low-dimensional latent representation data corresponding to the initial image, used to characterize the texture detail features and semantic features in the initial image. Specifically, the latent spatial feature data can be generated by encoding the initial image using a variational autoencoder network, a convolutional coding network, or a diffusion model coding module, used to map high-dimensional image data into the latent space to form a compact feature representation. This latent spatial feature data can effectively compress the original image information while preserving key texture details, color distribution features, and semantic context features, thereby reducing the complexity of data transmission and inference computation.

[0100] For example, a user terminal can invoke a locally deployed lightweight variational autoencoder (VAE) to perform feature encoding processing on an initial image. Specifically, the terminal device can utilize the acceleration capabilities of a Web Graphics Processing Unit (WebGPU) and a Neural Processing Unit (NPU) to load a pre-optimized VAE encoder model, completing the image encoding calculation in a background thread unnoticed by the user.

[0101] During the encoding process, the user terminal first performs size normalization, color space conversion, and pixel normalization on the initial image, and then inputs the processed red, green, and blue (RGB) image into the VAE encoder network. The VAE encoder uses a multi-layer convolutional network to perform step-by-step feature extraction and dimensionality reduction on the image, mapping high-dimensional pixel space data to a low-dimensional latent space to generate a corresponding latent feature tensor. This latent feature tensor can contain key representations such as texture detail features, color distribution features, and semantic context information.

[0102] S506: The above-mentioned structured feature data and the above-mentioned potential spatial feature data are fused to generate fused feature data.

[0103] The fused feature data can be used to simultaneously represent the spatial structural features and texture detail features of the initial image. Specifically, the fused feature data can be generated through feature concatenation, weighted fusion, attention-weighted fusion, or multi-channel mapping, enabling structural and texture information to be co-expressed in a unified feature space.

[0104] Specifically, structured feature data and latent spatial feature data can be concatenated along the channel dimension to form a joint feature tensor. The concatenated features can then be re-encoded using convolutional layers to achieve information compression and feature remodeling. Alternatively, structured feature data and latent spatial feature data can be linearly weighted and superimposed according to preset weight coefficients. Furthermore, a multi-layer convolutional mapping network can be used to map structured feature data and latent spatial feature data to a unified feature representation space, and then perform nonlinear fusion processing within that mapping space to generate fused feature data.

[0105] S508: The above-mentioned fused feature data is determined as the initial feature data corresponding to the above-mentioned initial image, and an image identifier corresponding to the above-mentioned initial feature data is generated.

[0106] The aforementioned image identifier can be a feature fingerprint information used to uniquely characterize the feature state of the current image generation request.

[0107] Specifically, based on the fused feature data, a preset feature fingerprint generation algorithm can be invoked to perform digest calculation on the fused feature data and generate the corresponding feature fingerprint value. For example, the aforementioned feature fingerprint generation algorithm may include at least one of a hash calculation algorithm, a cryptographic digest algorithm, or a perceptual hash algorithm.

[0108] Furthermore, the aforementioned image identifiers can be associated and bound with the corresponding initial feature data and processing instruction information for storage to form complete image generation request data, and the image generation request data can be sent to the server.

[0109] This application embodiment performs structured feature extraction and latent spatial feature encoding on the user terminal side, and then fuses the obtained features to generate fused feature data as initial feature data. This constructs a unified feature representation on the terminal side that combines spatial structural constraints and texture semantic expression capabilities. Therefore, while maintaining the structural stability of the target object, it enhances the ability to express texture details and semantic information, improving the overall quality and consistency of subsequent image generation results. Furthermore, by completing structured feature extraction and latent spatial feature encoding locally on the user terminal, the preprocessing computational burden on the cloud is effectively reduced, shortening the end-to-end image generation latency and optimizing the user experience.

[0110] Please see Figure 6 , Figure 6 This is a flowchart illustrating another image processing method provided in an embodiment of this application, applied to a server. For example... Figure 6 As shown, an image processing method may include at least: S602: Receive processing instruction information, initial feature data, and image identifier sent by the user terminal, wherein the initial feature data and the image identifier are data generated by the user terminal through preprocessing the initial image based on the processing instruction information.

[0111] Specifically, the server can receive image generation request data sent by the user terminal through the network communication module. This image generation request data includes processing instruction information, initial feature data, and image identifiers. Upon receiving the request data, the server first performs integrity verification and format parsing on the processing instruction information. Then, after confirming the legality and validity of the request data, the server decodes and verifies the initial feature data to ensure that no transmission anomalies have occurred. Finally, the server parses the image identifiers for subsequent cache matching and task indexing operations.

[0112] S604: Based on the preset diffusion model, perform image generation processing on the above processing instruction information, the above initial feature data and the above image identifier to obtain the target image corresponding to the above initial image.

[0113] The aforementioned preset diffusion model can be an image generation model built based on deep generative learning, used to gradually restore the noise signal to the target image data through a multi-step iterative denoising process. Specifically, the preset diffusion model can include a forward diffusion module and a reverse denoising generation module. The forward diffusion module is used to gradually add noise to the original image during the training phase, and the reverse denoising generation module is used to gradually restore the target image based on the noise distribution during the inference phase.

[0114] Optionally, the aforementioned preset diffusion model can be a conditional diffusion model. The conditional diffusion model incorporates the aforementioned processing instruction information, initial feature data, and image identifiers as conditional constraints during the generation process, thereby controlling the structural morphology, stylistic features, and detail representation of the generated image. The aforementioned preset diffusion model may include multiple neural network sub-modules, such as encoding networks, denoising prediction networks, attention networks, and conditional fusion networks, to achieve joint modeling of multi-source feature information.

[0115] Specifically, the aforementioned processing instructions can be converted into corresponding text semantic feature vectors, and the initial feature data can be mapped to the conditional input channel of the diffusion model to form a joint conditional feature input structure. Subsequently, the joint conditional features are input into a preset diffusion model to perform multi-time-step iterative denoising inference calculations, thereby gradually generating image feature representations that meet the target requirements, and further generating the target image.

[0116] S606: Return the target image to the user terminal.

[0117] Specifically, the server can also perform adaptive compression and resolution adjustment on the target image based on the user terminal's display resolution, network bandwidth status, and device performance parameters to maintain a balance between transmission efficiency and display effect.

[0118] Furthermore, the server can attach generated status information, timestamp information, and cache hit information when returning the target image to assist user terminals in result management and user experience optimization.

[0119] This application embodiment achieves efficient image generation through edge-cloud collaborative division of labor by receiving initial feature data and image identifiers preprocessed by the user terminal on the server side, and performing conditional image generation processing based on a preset diffusion model. In this application embodiment, the server side does not need to repeatedly perform computational operations such as structural feature extraction and latent spatial encoding, effectively reducing the cloud computing power burden and memory usage pressure, and improving processing capabilities and resource utilization efficiency in high-concurrency scenarios.

[0120] In one embodiment, such as Figure 7 In S604 above, the above-mentioned image generation processing based on the processing instruction information, the initial feature data and the image identifier according to the preset diffusion model to obtain the target image corresponding to the initial image may include the following steps: S702: Input the above processing instruction information, the above initial feature data and the above image identifier into the above preset diffusion model to perform denoising processing at multiple time steps.

[0121] The aforementioned multiple time steps can be consecutive time steps or computational stages included in the denoising and generation process of the preset diffusion model during the inference phase, used to gradually restore random noise features to target image features. Each time step corresponds to one feature update computation process of the diffusion model, and multiple time steps together constitute a complete image generation process.

[0122] Specifically, the server can convert processing instruction information into corresponding semantic feature representations using a pre-defined text encoding model, and map the initial feature data to the conditional input channel of the diffusion model. Simultaneously, image identifiers are bound and indexed with their corresponding feature data. Subsequently, the aforementioned processing instruction information, initial feature data, and image identifiers are input as joint conditions into the pre-defined diffusion model to initialize the diffusion denoising iterative process. The aforementioned text encoding model can be built based on deep learning and used for semantic understanding and feature extraction of text sequences, thereby generating semantic feature representations that characterize the semantic information of the text.

[0123] S704: For each of the above multiple time steps, obtain the current intermediate feature data corresponding to the current time step, and calculate the feature similarity between the current intermediate feature data and the historical intermediate feature data of the previous time step.

[0124] The aforementioned intermediate feature data can be the stage-wise feature representation data generated by the preset diffusion model at each time step, used to characterize the potential image representation under the current denoising state. This intermediate feature data can be in the form of a multi-dimensional feature tensor, containing current image structural information, texture information, and semantic information.

[0125] It is understandable that the aforementioned historical intermediate feature data can be intermediate feature data generated and cached in the previous time step or multiple historical time steps, used as a reference for calculation in the current time step. This historical intermediate feature data can reflect the evolutionary trajectory during the image generation process.

[0126] Optionally, the aforementioned feature similarity is used to characterize the degree of similarity between the current intermediate feature data and the historical intermediate feature data, and can reflect the magnitude of changes in the image generation state. The aforementioned feature similarity can be calculated based on methods such as cosine similarity, Euclidean distance similarity, or correlation coefficient.

[0127] Specifically, after completing the denoising prediction at each time step, the server can obtain the currently generated intermediate feature data and read the corresponding historical intermediate feature data from the cache module. Subsequently, based on a preset similarity calculation algorithm, the server calculates the similarity between the current intermediate feature data and the historical intermediate feature data to generate the corresponding feature similarity value.

[0128] S706: If the similarity of the above features is greater than the preset similarity threshold, perform the above denoising processing in the current time step based on the above historical intermediate feature data.

[0129] The aforementioned preset similarity threshold can be used to determine the degree of change in intermediate features. When the feature similarity is greater than the preset similarity threshold, it indicates that the current feature has changed relatively little; when the feature similarity is not greater than the preset similarity threshold, it indicates that the current feature has changed relatively much. It is understood that the preset similarity threshold can be set and dynamically adjusted based on historical statistical data or experimental results. For example, the preset similarity threshold can be 95%.

[0130] Optionally, denoising can be a computational process in which the diffusion model predicts and corrects the noise distribution during the backward inference process, used to gradually reduce the noise component in the features so that the image features converge to the true distribution.

[0131] Specifically, when the server detects that the feature similarity at the current time step is greater than a preset similarity threshold, it can determine that the current feature change is relatively small. In this case, the server can directly call historical intermediate feature data to participate in the current denoising and prediction calculation, skipping some attention and convolution calculation modules, thereby reducing the amount of repetitive computation.

[0132] S708: If the similarity of the above features is not greater than the preset similarity threshold, perform denoising processing based on the above processing instruction information and the above current intermediate feature data to generate new intermediate feature data.

[0133] Among them, the new intermediate feature data can be the intermediate feature data generated and stored after complete denoising calculation in the current time step, which can be used for similarity matching and calculation reuse in subsequent time steps.

[0134] Specifically, when the feature similarity is not greater than a preset similarity threshold, it can be determined that the current feature state has changed significantly. At this time, the server executes the complete diffusion denoising calculation process based on the processing instruction information and the current intermediate feature data, generates new intermediate feature data, and writes the new intermediate feature data into the cache module.

[0135] S710: Under the condition that the preset iteration stopping condition is met, based on the new intermediate feature data, the target image corresponding to the initial image is generated by decoding through the preset diffusion model.

[0136] The aforementioned preset iteration stopping conditions are used to control the termination time of the diffusion inference process. Specifically, they may include at least one condition such as reaching the preset maximum number of iterations, the current noise intensity being lower than a threshold, the feature change amplitude being less than a threshold, or the generated quality index meeting the requirements.

[0137] Optionally, the above decoding process can be a process of mapping the latent spatial features finally output by the diffusion model back to the pixel spatial image. Specifically, it can be implemented through a decoding network or a generative decoding module to restore the latent features into visualized image data.

[0138] Specifically, when the current time step is detected to meet the preset iteration stop condition, the diffusion denoising iteration process is terminated, and the final generated intermediate feature data is input into the decoding module to perform decoding processing, restoring the latent spatial features to the target image data, thereby generating the target image corresponding to the initial image mentioned above.

[0139] In the denoising iteration process across multiple time steps, this application calculates the similarity between the current intermediate feature data and historical intermediate feature data. When a small feature change is detected, historical intermediate feature data is directly reused in the current denoising calculation, thus skipping some repetitive attention or convolution calculation modules, significantly reducing inference computation and server computing power consumption. When a significant feature change is detected, the complete denoising inference process is executed, ensuring that the image structure and detail representation remain unaffected. Therefore, the diffusion model can dynamically adjust the computation path based on feature changes during the inference phase, improving image generation efficiency, shortening response latency, and effectively reducing energy consumption and resource usage.

[0140] The present solution will be described below with reference to specific embodiments.

[0141] Figure 8 A detailed flowchart of an image processing method provided for an exemplary embodiment of this application is shown below. Figure 8 The embodiments of this application adopt an edge-cloud collaborative image processing architecture, and the overall process can be divided into two stages: the preprocessing stage on the user terminal side and the generation stage on the server side.

[0142] First, the user can specify the initial image and corresponding processing instructions on the user terminal. The user terminal interface displays a "Generating..." message to maintain the continuity of user interaction. Then, the process enters the first stage: edge-side preprocessing. The terminal determines the target processing strategy based on the processing instructions and selects the corresponding processing mode. In Mode A, the user terminal calls a local computer vision algorithm to execute the first processing strategy, performing structured feature extraction on the initial image, such as extracting edge feature maps, contour structure maps, or depth structure maps, generating structured feature data. In Mode B, the user terminal calls a local VAE encoder to execute the second processing strategy, mapping the initial image to a low-dimensional latent space, generating latent space feature data. When the target processing strategy is a combined strategy, the user terminal can execute the two processing methods in parallel and fuse the structured feature data and latent space feature data to generate fused feature data. Subsequently, the user terminal calculates the corresponding image identifier based on the generated feature data. This image identifier can be a feature fingerprint generated based on feature summarization, used to uniquely identify the current image generation request status.

[0143] Subsequently, after completing the preprocessing on the client side, the user terminal packages the processing instruction information, initial feature data, and image identifiers into image request data and sends it to the server. After receiving the image generation request data, the server can first check whether there is a corresponding full result cache to improve the hit efficiency; if there is no cached result, it enters the diffusion inference stage.

[0144] During the server-side generation phase, the server can execute a step-by-step denoising iterative process based on a preset diffusion model. At each time step, the current intermediate feature data is generated, and its feature similarity is calculated with the historical intermediate feature data from the previous time step. When the feature similarity is greater than a preset similarity threshold, the server directly reuses the historical intermediate feature data in the current time step's calculation, skipping some repetitive attention or convolution calculations. When the feature similarity is not greater than the preset threshold, the complete denoising calculation process is executed, and new intermediate feature data is generated and written to the cache module. This dynamic stepping strategy achieves adaptive computational optimization during the diffusion inference process. After meeting the preset iteration stopping condition, the server uses the decoding module to restore the final intermediate feature data to the target image and returns the target image to the user terminal to complete the image generation process.

[0145] It should be noted that the aforementioned edge-side preprocessing and server-side dynamic step-by-step optimization processes are all executed automatically in background threads or independent computing modules, requiring no additional user intervention or waiting for intermediate results. The terminal device immediately displays a "Generating..." status after the user clicks "Generate," but the structured feature extraction, latent spatial encoding, feature fusion, and image label generation are all completed silently locally, without altering the user's existing workflow. Furthermore, the server-side diffusion iteration optimization and feature cache reuse processes are also executed automatically within the system, without exposing specific computational steps or intermediate states to the user. The entire image generation process appears to the user as a single, continuous generation operation, with no interface transitions or additional confirmation steps, achieving seamless optimization that improves both generation efficiency and the smoothness and consistency of the user experience.

[0146] This application's embodiments construct an edge-cloud collaborative image processing architecture, preprocessing operations such as structured feature extraction and latent spatial encoding are performed locally on the user terminal, effectively distributing the preprocessing load on the cloud and reducing cloud memory usage and queuing time. Furthermore, by introducing a dynamic computation reuse method based on feature similarity during the diffusion model inference process on the server side, adaptive optimization of the denoising iteration process is achieved. This significantly reduces redundant computation steps while improving image generation quality, lowering inference latency and computational power consumption. Therefore, this application's embodiments significantly shorten end-to-end generation time, improve the real-time performance and concurrent processing capabilities of image generation, and enhance the overall user experience while maintaining the same user operation flow.

[0147] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps.

[0148] Based on the inventive concept of the above image processing method, such as Figure 9 As shown, this application embodiment also provides a first image processing apparatus 900 for implementing the image processing method described above. The first image processing apparatus 900 includes: The first acquisition module 910 is used to acquire the initial image and the processing instruction information corresponding to the initial image; The first processing module 920 is used to preprocess the initial image based on the above processing instruction information to generate initial feature data and image identifiers corresponding to the initial image. The first sending module 930 is used to send the processing instruction information, the initial feature data and the image identifier to the server, so that the server performs image generation processing based on the processing instruction information, the initial feature data and the image identifier to obtain the target image corresponding to the initial image. The first receiving module 940 is used to receive the target image returned by the server.

[0149] In one possible implementation, the first processing module 920 is specifically used for: Based on the above processing instruction information, a corresponding target processing strategy is determined. The target processing strategy includes a first processing strategy and / or a second processing strategy. The first processing strategy is used to perform structured feature extraction processing on the above initial image, and the second processing strategy is used to perform feature encoding processing on the above initial image. Based on the above target processing strategy and the above processing instruction information, the above initial image is preprocessed to generate the initial feature data and image identifier corresponding to the above initial image.

[0150] In one possible implementation, the first processing module 920 is specifically used for: The above processing instruction information is semantically parsed to generate the corresponding processing task type; Determine whether the preset mapping relationship includes the preset processing strategy corresponding to the above-mentioned processing task type; If the preset mapping relationship includes the preset processing strategy corresponding to the above processing task type, the preset processing strategy corresponding to the above processing task type shall be determined as the target processing strategy. If the preset processing strategy corresponding to the above processing task type is not included in the above preset mapping relationship, the image attribute information of the above initial image is obtained, and the corresponding target processing strategy is determined based on the above image attribute information.

[0151] In one possible implementation, the first processing module 920 is specifically used for: Based on the above image attribute information, the above initial image is subjected to image content evaluation processing to generate image structure evaluation parameters and image texture evaluation parameters corresponding to the above initial image. The above image structure evaluation parameters are used to characterize the completeness of spatial structure feature information in the above initial image, and the above image texture evaluation parameters are used to characterize the complexity of texture detail feature information in the above initial image. Based on the above image structure evaluation parameters, the above image texture evaluation parameters, and the preset evaluation parameter thresholds, the corresponding target processing strategy is determined.

[0152] In one possible implementation, the aforementioned preset evaluation parameter thresholds include a first preset threshold and a second preset threshold; the first processing module 920 is specifically used for: If the above image structure evaluation parameter is greater than the first preset threshold and the above image texture evaluation parameter is less than the second preset threshold, the above first processing strategy shall be used as the corresponding target processing strategy. If the above image structure evaluation parameter is not greater than the above first preset threshold and the above image texture evaluation parameter is greater than the above second preset threshold, the above second processing strategy shall be used as the corresponding target processing strategy. If the above image structure evaluation parameter is greater than the above first preset threshold and the above image texture evaluation parameter is not less than the above second preset threshold, the above first processing strategy and the above second processing strategy are used as the corresponding target processing strategy. If the image structure evaluation parameters are not greater than the first preset threshold and the image texture evaluation parameters are not greater than the second preset threshold, the second processing strategy will be used as the corresponding target processing strategy.

[0153] In one possible implementation, where the target processing strategy includes the first processing strategy and the second processing strategy, the first processing module 920 is specifically used for: The initial image above is subjected to structured feature extraction processing to generate structured feature data; The initial image is subjected to feature encoding processing to generate latent spatial feature data; The structured feature data and the potential spatial feature data are fused together to generate fused feature data. The aforementioned fused feature data is determined as the initial feature data corresponding to the aforementioned initial image, and an image identifier corresponding to the aforementioned initial feature data is generated.

[0154] In one possible implementation, the first acquisition module 910 is specifically used for: Obtain the initial image determined by the user, and obtain the task description information determined by the user for the initial image; The above task description information is semantically parsed to generate corresponding task semantic feature information; Based on the above task semantic feature information and the preset task template library, a matching analysis is performed to determine the target processing task type and the corresponding set of processing parameters. The above processing instruction information is generated based on the above target processing task type and the above processing parameter set.

[0155] Based on the inventive concept of the other image processing method mentioned above, such as Figure 10 As shown, this application embodiment also provides a second image processing apparatus 1000 for implementing another image processing method involved above. The second image processing apparatus 1000 includes: The second receiving module 1010 is used to receive processing instruction information, initial feature data and image identifier sent by the user terminal, wherein the initial feature data and the image identifier are data generated by the user terminal by preprocessing the initial image based on the processing instruction information. The second processing module 1020 is used to perform image generation processing on the above-mentioned processing instruction information, the above-mentioned initial feature data and the above-mentioned image identifier based on a preset diffusion model, so as to obtain the target image corresponding to the above-mentioned initial image. The second sending module 1030 is used to return the target image to the user terminal.

[0156] In one possible implementation, the second processing module 1020 is specifically used for: The above processing instruction information, the above initial feature data and the above image identifier are input into the above preset diffusion model to perform denoising processing at multiple time steps; For each of the above multiple time steps, obtain the current intermediate feature data corresponding to the current time step, and calculate the feature similarity between the current intermediate feature data and the historical intermediate feature data of the previous time step; If the similarity of the above features is greater than the preset similarity threshold, the above denoising processing in the current time step is performed based on the above historical intermediate feature data; If the similarity of the above features is not greater than the above preset similarity threshold, denoising is performed based on the above processing instruction information and the above current intermediate feature data to generate new intermediate feature data; Under the condition that the preset iteration stopping condition is met, the target image corresponding to the initial image is generated by decoding the new intermediate feature data through the preset diffusion model.

[0157] The division of modules in the first image processing device / second image processing device described above is for illustrative purposes only. In other embodiments, the first image processing device / second image processing device can be divided into different modules as needed to complete all or part of the functions of the first image processing device / second image processing device. The implementation of each module in the first image processing device / second image processing device provided in this application embodiment can be in the form of a computer program. This computer program can run on a terminal or server. The program modules constituted by this computer program can be stored in the memory of the terminal or server. When the computer program is executed by a processor, it implements all or part of the steps of the image processing method described in the embodiments of this application.

[0158] This application also provides an electronic device, which can be a server, and its internal structure diagram can be as follows: Figure 11 As shown, this electronic device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. The processor executes computer programs to implement an image processing method.

[0159] Those skilled in the art will understand that Figure 11 The structures shown are merely block diagrams of some structures related to the embodiments of this application and do not constitute a limitation on the electronic devices to which the embodiments of this application are applied. Specific electronic devices may include more or fewer components than those shown in the figures, or combine certain components, or have different component arrangements.

[0160] In one possible implementation, a computer storage medium is provided that stores instructions, which, when executed on a computer or processor, cause the computer or processor to perform one or more steps in the above embodiments. If the constituent modules of the above-described electronic device are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium.

[0161] In one possible implementation, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0162] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer storage medium or transmitted through the computer storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The aforementioned available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Versatile Discs (DVDs)), or semiconductor media (e.g., Solid State Disks (SSDs)).

[0163] It should be noted that the information (including but not limited to processing instruction information, image attribute information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in the embodiments of this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions. For example, the initial feature data and intermediate feature data involved in this application were obtained under full authorization.

[0164] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks. Unless otherwise specified, the technical features of this embodiment and its implementation can be combined arbitrarily.

[0165] The above-described embodiments are merely preferred embodiments of the present application and are not intended to limit the scope of the present application. Various modifications and improvements made by those skilled in the art to the technical solutions of the present application without departing from the design spirit of the present application should fall within the protection scope defined by the claims.

[0166] The foregoing has described specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in a different order than that shown in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. An image processing method, applied to a user terminal, comprising: Obtain the initial image and the corresponding processing instruction information for the initial image; Based on the processing instruction information, the initial image is preprocessed to generate initial feature data and image identifiers corresponding to the initial image; The processing instruction information, the initial feature data, and the image identifier are sent to the server, so that the server performs image generation processing based on the processing instruction information, the initial feature data, and the image identifier to obtain the target image corresponding to the initial image; Receive the target image returned by the server.

2. The method as described in claim 1, wherein preprocessing the initial image based on the processing instruction information to generate initial feature data and image identifiers corresponding to the initial image includes: Based on the processing instruction information, a corresponding target processing strategy is determined. The target processing strategy includes a first processing strategy and / or a second processing strategy. The first processing strategy is used to perform structured feature extraction processing on the initial image, and the second processing strategy is used to perform feature encoding processing on the initial image. The initial image is preprocessed based on the target processing strategy and the processing instruction information to generate initial feature data and image identifiers corresponding to the initial image.

3. The method as described in claim 2, wherein determining the corresponding target processing strategy based on the processing instruction information includes: The processing instruction information is semantically parsed to generate the corresponding processing task type; Determine whether the preset mapping relationship includes the preset processing strategy corresponding to the processing task type; If the preset mapping relationship includes a preset processing strategy corresponding to the processing task type, the preset processing strategy corresponding to the processing task type is determined as the target processing strategy. If the preset processing strategy corresponding to the processing task type is not included in the preset mapping relationship, the image attribute information of the initial image is obtained, and the corresponding target processing strategy is determined based on the image attribute information.

4. The method as described in claim 3, wherein determining the corresponding target processing strategy based on the image attribute information includes: Based on the image attribute information, the initial image is subjected to image content evaluation processing to generate image structure evaluation parameters and image texture evaluation parameters corresponding to the initial image. The image structure evaluation parameters are used to characterize the completeness of spatial structure feature information in the initial image, and the image texture evaluation parameters are used to characterize the complexity of texture detail feature information in the initial image. Based on the image structure evaluation parameters, the image texture evaluation parameters, and the preset evaluation parameter thresholds, the corresponding target processing strategy is determined.

5. The method as described in claim 4, wherein the preset evaluation parameter threshold includes a first preset threshold and a second preset threshold; the step of determining the corresponding target processing strategy based on the image structure evaluation parameter, the image texture evaluation parameter, and the preset evaluation parameter threshold includes: If the image structure evaluation parameter is greater than a first preset threshold and the image texture evaluation parameter is less than a second preset threshold, the first processing strategy is used as the corresponding target processing strategy. If the image structure evaluation parameter is not greater than the first preset threshold and the image texture evaluation parameter is greater than the second preset threshold, the second processing strategy shall be used as the corresponding target processing strategy. If the image structure evaluation parameter is greater than the first preset threshold and the image texture evaluation parameter is not less than the second preset threshold, the first processing strategy and the second processing strategy are used as the corresponding target processing strategies. If the image structure evaluation parameter is not greater than the first preset threshold and the image texture evaluation parameter is not greater than the second preset threshold, the second processing strategy is used as the corresponding target processing strategy.

6. The method of claim 2, wherein when the target processing strategy includes the first processing strategy and the second processing strategy, the step of preprocessing the initial image based on the target processing strategy and the processing instruction information to generate initial feature data and image identifiers corresponding to the initial image includes: The initial image is subjected to structured feature extraction processing to generate structured feature data; The initial image is subjected to feature encoding processing to generate latent spatial feature data; The structured feature data and the latent spatial feature data are fused together to generate fused feature data; The fused feature data is determined as the initial feature data corresponding to the initial image, and an image identifier corresponding to the initial feature data is generated.

7. The method as described in claim 2, wherein obtaining the initial image and the processing instruction information corresponding to the initial image includes: Obtain the initial image determined by the user, and obtain the task description information determined by the user for the initial image; The task description information is semantically parsed to generate corresponding task semantic feature information; Based on the task semantic feature information, a matching analysis is performed with a preset task template library to determine the target processing task type and the corresponding set of processing parameters. The processing instruction information is generated based on the target processing task type and the processing parameter set.

8. An image processing method, applied to a server, comprising: The system receives processing instruction information, initial feature data, and image identifiers sent by a user terminal, wherein the initial feature data and the image identifiers are data generated by the user terminal through preprocessing an initial image based on the processing instruction information. Based on a preset diffusion model, the processing instruction information, the initial feature data, and the image identifier are processed to generate an image, thereby obtaining the target image corresponding to the initial image. The target image is returned to the user terminal.

9. The method as described in claim 8, wherein the step of performing image generation processing on the processing instruction information, the initial feature data, and the image identifier based on a preset diffusion model to obtain the target image corresponding to the initial image includes: The processing instruction information, the initial feature data, and the image identifier are input into the preset diffusion model to perform denoising processing at multiple time steps; For each of the multiple time steps, obtain the current intermediate feature data corresponding to the current time step, and calculate the feature similarity between the current intermediate feature data and the historical intermediate feature data of the previous time step; If the feature similarity is greater than a preset similarity threshold, denoising processing is performed at the current time step based on the historical intermediate feature data. If the feature similarity is not greater than the preset similarity threshold, denoising is performed based on the processing instruction information and the current intermediate feature data to generate new intermediate feature data; If the preset iteration stopping condition is met, the target image corresponding to the initial image is generated by decoding the new intermediate feature data through the preset diffusion model.

10. An image processing system, comprising a user terminal and a server; wherein, The user terminal is used to perform the method as described in any one of claims 1-7, and the server is used to perform the method as described in any one of claims 8-9.

11. An image processing apparatus, applied to a user terminal, comprising: The first acquisition module is used to acquire an initial image and processing instruction information corresponding to the initial image; The first processing module is used to preprocess the initial image based on the processing instruction information to generate initial feature data and image identifiers corresponding to the initial image. The first sending module is used to send the processing instruction information, the initial feature data, and the image identifier to the server, so that the server performs image generation processing based on the processing instruction information, the initial feature data, and the image identifier to obtain the target image corresponding to the initial image; The first receiving module is used to receive the target image returned by the server.

12. An image processing apparatus, applied to a server, comprising: The second receiving module is used to receive processing instruction information, initial feature data and image identifier sent by the user terminal, wherein the initial feature data and the image identifier are data generated by the user terminal by preprocessing the initial image based on the processing instruction information; The second processing module is used to perform image generation processing on the processing instruction information, the initial feature data and the image identifier based on a preset diffusion model to obtain the target image corresponding to the initial image; The second sending module is used to return the target image to the user terminal.

13. An electronic device, comprising: Processor and memory; The processor is connected to the memory; The memory is used to store executable program code; The processor runs a program corresponding to the executable program code stored in the memory to perform the method as described in any one of claims 1-7, or to perform the method as described in any one of claims 8-9.

14. A computer storage medium storing a plurality of instructions adapted for loading by a processor and executing the method of any one of claims 1-7, or the instructions adapted for loading by a processor and executing the method of any one of claims 8-9.

15. A computer program product comprising instructions which, when run on a computer or processor, cause the computer or processor to perform the method as claimed in any one of claims 1-7, or cause the computer or processor to perform the method as claimed in any one of claims 8-9.

Images