An image reasoning method, apparatus, device, and storage medium

Abstract

This invention discloses an image inference method, apparatus, device, and storage medium applied to a diffusion model. The method includes: acquiring an initial image and determining the current time step; generating initial noise based on the initial image and the current time step; sampling based on the current time step and the initial noise to generate sampling features, wherein the sampling features include partial sampling features and complete sampling features; and generating a final image based on the current time step and the sampling features. The method generates initial noise by determining a linear combination of targets using a preset knowledge base, thereby skipping the early sampling steps of the diffusion denoising process. By determining whether to perform partial or complete inference sampling based on different current time steps and initial noise, and by introducing cached features at partial time steps to avoid performing complete inference, only a small portion of the inference is needed to achieve an effect similar to complete inference, saving inference time and computational overhead, thus accelerating the diffusion model.

Description

Technical Field

[0001] This invention relates to the field of image diffusion technology, and in particular to an image reasoning method, apparatus, device, and storage medium. Background Technology

[0002] With the popularization of diffusion models, their excellent generative capabilities have been widely applied in various fields. Diffusion models simulate the image generation process through a series of random diffusion steps. Their core concept is to start with random noise and gradually refine it until it resembles a sample from the target distribution. In the reverse reasoning process, the diffusion model continuously denoises a random, completely noisy image, ultimately generating a completely new image.

[0003] Currently, the main techniques for reducing inference costs focus on reducing the number of inference steps and using techniques such as distillation, quantization, and pruning to reduce the model inference overhead per step. However, methods such as distillation and quantization to obtain lightweight compressed models often require large-scale retraining of the model. Although pruning can reduce the cost of creating new lightweight models, it still requires a large amount of computational resources and time, resulting in high inference overhead and low efficiency. Summary of the Invention

[0004] This invention provides an image reasoning method, apparatus, device, and storage medium to accelerate the reasoning efficiency of diffusion models and improve the speed of image reasoning by caching intermediate states.

[0005] According to one aspect of the present invention, an image reasoning method is provided, applied to a diffusion model, the method comprising:

[0006] Acquire the initial image and determine the current time step; generate initial noise based on the initial image and the current time step.

[0007] Sampling is performed based on the current time step and initial noise to generate sampling features, which include partial sampling features and complete sampling features;

[0008] The final image is generated based on the current time step and sampling features.

[0009] Optionally, generating initial noise based on the initial image and the current time step includes: generating early noise based on the initial image when the current time step is 0; generating historical sampling features by performing several inference sampling steps based on the early noise and the current time step; and matching the historical sampling features with a preset knowledge base when the current time step reaches a specified time step to generate initial noise.

[0010] Optionally, the initial noise is generated by matching historical sampling features with a preset knowledge base, including: determining whether the preset knowledge base contains a specified number of features that are similar to the historical sampling features; if so, determining each linear combination of the specified number of features, taking the linear combination with the highest similarity to the historical sampling features as the target linear combination, determining the target weight corresponding to the target linear combination, calling the historical intermediate state of the trajectory of the target linear combination at a specified stage, and combining the historical intermediate state with the target weight to generate the initial noise; otherwise, the initial noise is generated by randomly initializing based on the initial image.

[0011] Optionally, sampling is performed based on the current time step and initial noise to generate sampled features, including: determining the cache feature invocation status based on the current time step, wherein the cache feature invocation status includes invoking the cache feature and not invoking the cache feature; when the cache feature invocation status is invoking the cache feature, the cache feature is obtained, and partial inference sampling is performed based on the initial noise and the cache feature to generate partial sampled features; when the cache feature invocation status is not invoking the cache feature, full sampling is performed based on the initial noise to generate full sampled features.

[0012] Optionally, after performing full sampling based on the initial noise, the method further includes: determining intermediate features generated during the full sampling process; and storing the intermediate features to generate cached features.

[0013] Optionally, the cache feature invocation status is determined based on the current time step, including: obtaining a preset time step sequence, wherein the preset time step sequence stores the time steps that need to be fully sampled; determining whether the current time step is in the preset time step sequence, if so, determining that the cache feature invocation status is not to invoke the cache feature; otherwise, determining that the cache feature invocation status is to invoke the cache feature.

[0014] Optionally, generating the final image based on the current time step and sampling features includes: determining whether the current time step has reached a preset time step threshold; if so, generating the final image based on the sampling features; otherwise, performing inference sampling for the next time step.

[0015] According to another aspect of the present invention, an image inference apparatus is provided for use in a diffusion model, the apparatus comprising:

[0016] The initial noise generation module is used to acquire the initial image, determine the current time step, and generate initial noise based on the initial image and the current time step.

[0017] The sampling feature generation module is used to sample based on the current time step and initial noise to generate sampling features, wherein the sampling features include partial sampling features and complete sampling features;

[0018] The final image generation module is used to generate the final image based on the current time step and sampling features.

[0019] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0020] At least one processor;

[0021] and a memory communicatively connected to the at least one processor;

[0022] The memory stores a computer program that can be executed by the at least one processor, which is then executed by the at least one processor to enable the at least one processor to perform an image reasoning method according to any embodiment of the present invention.

[0023] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement an image reasoning method according to any embodiment of the present invention.

[0024] The technical solution of this invention generates initial noise by determining the target linear combination through a preset knowledge base, thereby skipping the early sampling step of the diffusion denoising process. By determining whether to perform partial inference sampling or full inference sampling based on different current time steps and initial noise, and by introducing cached features at some time steps to avoid performing full inference, only a small part of the inference is needed to achieve the effect of near-full inference, saving inference time and computational overhead, thereby achieving the purpose of accelerating the diffusion model.

[0025] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a flowchart of an image reasoning method provided in Embodiment 1 of the present invention;

[0028] Figure 2 This is a flowchart of another image reasoning method provided in Embodiment 2 of the present invention;

[0029] Figure 3 This is a schematic diagram of the structure of an image reasoning device according to Embodiment 3 of the present invention;

[0030] Figure 4 This is a schematic diagram of the structure of an electronic device that implements an image reasoning method according to an embodiment of the present invention. Detailed Implementation

[0031] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0033] Example 1

[0034] Figure 1 The flowchart of an image inference method provided in Embodiment 1 of the present invention is applicable to situations where a diffusion model is used for reverse inference of an image. This method can be executed by an image inference device, which can be implemented in hardware and / or software and can be configured in a computer controller. Figure 1 As shown, the method includes:

[0035] S110. Obtain the initial image and determine the current time step. Generate initial noise based on the initial image and the current time step.

[0036] The diffusion model simulates the image generation process through a series of random diffusion steps. Its core concept is to start with random noise and gradually refine it until it resembles a sample from the target distribution. The diffusion process includes forward and backward reasoning; this implementation specifically applies to the backward reasoning process of the diffusion model. In backward reasoning, the diffusion model continuously denoises a random, completely noisy image, eventually generating a completely new image. The initial image is the starting point of the entire image reasoning process; it is existing image data, which can be any type of image, such as a photograph or painting. In the diffusion model, the initial image provides the basic information and features for subsequent reasoning. Time steps represent different stages in the evolution of the image from its initial state to its final state. As the time steps increase, noise is gradually removed from the image; different time steps correspond to different levels of noise and changes in image features. The initial noise is the initial sampling point generated based on the initial image and the current time step. This implementation mainly includes two parts: a predicted intermediate state part that directly skips early reasoning sampling steps by retrieving the intermediate state with the highest similarity from a preset knowledge base, and a cached intermediate state part that uses store-and-call calls to continuously approximate intermediate states to avoid redundant calculations.

[0037] Optionally, generating initial noise based on the initial image and the current time step includes: generating early noise based on the initial image when the current time step is 0; generating historical sampling features by performing several inference sampling steps based on the early noise and the current time step; and matching the historical sampling features with a preset knowledge base when the current time step reaches a specified time step to generate initial noise.

[0038] Here, a time step represents a different stage in the evolution of an image from its initial state to its final state. If the current time step is 0, it means that the image is in its initial state. When the current time step is determined to be 0, an early noise can be obtained based on the initial image described by the prompt. When the current time step reaches a specified time step, the controller can further acquire historical sampling features, which refer to the sampling features of the previous few steps. Then, the controller matches the historical sampling features with a preset knowledge base to generate the corresponding initial noise.

[0039] Optionally, the initial noise is generated by matching historical sampling features with a preset knowledge base, including: determining whether the preset knowledge base contains a specified number of features that are similar to the historical sampling features; if so, determining each linear combination of the specified number of features, taking the linear combination with the highest similarity to the historical sampling features as the target linear combination, determining the target weight corresponding to the target linear combination, calling the historical intermediate state of the trajectory of the target linear combination at a specified stage, and combining the historical intermediate state with the target weight to generate the initial noise; otherwise, the initial noise is generated by randomly initializing based on the initial image.

[0040] Specifically, the pre-built knowledge base is a collection of various feature information stored in advance. The controller can search the database to find a specified number of features that are most similar to historical sampled features. If similar features exist in the knowledge base, the controller will further determine the linear combinations of the specified number of features. A linear combination is a new feature obtained by weighted summation of multiple features. Different linear combinations can be obtained by combining different weights. Then, the similarity between each linear combination and the historical sampled features is calculated, and the linear combination with the highest similarity is determined as the target linear combination. After determining the target linear combination, the target weights corresponding to the combination also need to be determined. The weights determine the contribution of each feature in the combination. Here, the controller refers to the computer controller that performs image reasoning.

[0041] In the diffusion model, the diffusion process is typically a gradual trajectory with corresponding intermediate states at each stage. The designated stage refers to the later part of the trajectory. The controller retrieves historical intermediate states from the later part of the trajectory containing the target linear combination, combines these historical intermediate states with the target weights, and uses this combination as the initial point for the remaining steps of the sampling process—the initial noise. By utilizing a pre-set knowledge base for retrieval, existing information can be used to generate the initial noise, thereby saving computational overhead and improving inference efficiency.

[0042] Specifically, if the preset knowledge base does not contain a specified number of features that are similar to the historical sampling features, then the information in the knowledge base cannot be used to generate initial noise. In this case, the controller will randomly initialize and generate initial noise based on the initial image.

[0043] S120. Sample based on the current time step and initial noise to generate sampling features, wherein the sampling features include partial sampling features and complete sampling features.

[0044] In image inference, sampling is performed based on the current time step and initial noise, with the aim of generating sampled features. Through sampling, a subset of features can be extracted from the noise distribution for use in generating subsequent images. Sampled features include partial sampled features and complete sampled features.

[0045] S130. Generate the final image based on the current time step and sampling features.

[0046] Optionally, generating the final image based on the current time step and sampling features includes: determining whether the current time step has reached a preset time step threshold; if so, generating the final image based on the sampling features; otherwise, performing inference sampling for the next time step.

[0047] It's important to note that in the diffusion model, the progression of time steps represents the gradual evolution of the image from an initial noisy state to a final clear image state. The preset time step threshold is a pre-defined time step value that marks a stage where the image generation process is nearing completion. By determining whether the current time step has reached the preset time step threshold, the progress of image generation can be determined.

[0048] Specifically, when the current time step reaches a preset time step threshold, the controller can generate the final image based on the sampled features. The sampled features are image features obtained through sampling in previous time steps. Generating the final image typically involves integrating and processing the sampled features. For example, this may include merging partial and complete sampled features, performing weighted summation of features, and mapping through a neural network to obtain the final clear image.

[0049] It is known that if the current time step has not reached the preset time step threshold, it means that the image generation process is not yet complete and inference sampling needs to continue to the next time step. The inference sampling process of the next time step is similar to that of the previous time step. Based on the current image state (which may be a partially denoised image or a noisy image) and the current time step, sampling is performed to generate new sampling features. This process is repeated continuously. As the time steps advance, the image gradually becomes clearer, and the sampling features become closer and closer to the features of the final image.

[0050] The technical solution of this invention generates initial noise by determining the target linear combination through a preset knowledge base, thereby skipping the early sampling step of the diffusion denoising process. By determining whether to perform partial inference sampling or full inference sampling based on different current time steps and initial noise, and by introducing cached features at some time steps to avoid performing full inference, only a small part of the inference is needed to achieve the effect of near-full inference, saving inference time and computational overhead, thereby achieving the purpose of accelerating the diffusion model.

[0051] Example 2

[0052] Figure 2 This is a flowchart of an image reasoning method provided in Embodiment 2 of the present invention. This embodiment adds a specific process for generating sampled features based on the current time step and initial noise, building upon Embodiment 1. The specific content of steps S210 and S250 is largely the same as steps S110 and S130 in Embodiment 1, and therefore will not be repeated in this embodiment. Figure 2 As shown, the method includes:

[0053] S210. Obtain the initial image and determine the current time step. Generate initial noise based on the initial image and the current time step.

[0054] Optionally, generating initial noise based on the initial image and the current time step includes: generating early noise based on the initial image when the current time step is 0; generating historical sampling features by performing several inference sampling steps based on the early noise and the current time step; and matching the historical sampling features with a preset knowledge base when the current time step reaches a specified time step to generate initial noise.

[0055] Optionally, the initial noise is generated by matching historical sampling features with a preset knowledge base, including: determining whether the preset knowledge base contains a specified number of features that are similar to the historical sampling features; if so, determining each linear combination of the specified number of features, taking the linear combination with the highest similarity to the historical sampling features as the target linear combination, determining the target weight corresponding to the target linear combination, calling the historical intermediate state of the trajectory of the target linear combination at a specified stage, and combining the historical intermediate state with the target weight to generate the initial noise; otherwise, the initial noise is generated by randomly initializing based on the initial image.

[0056] S220. Determine the cache feature invocation status based on the current time step, where cache feature invocation status includes invoking cache features and not invoking cache features.

[0057] Optionally, the cache feature invocation status is determined based on the current time step, including: obtaining a preset time step sequence, wherein the preset time step sequence stores the time steps that need to be fully sampled; determining whether the current time step is in the preset time step sequence, if so, determining that the cache feature invocation status is not to invoke the cache feature; otherwise, determining that the cache feature invocation status is to invoke the cache feature.

[0058] The preset time step sequence is a collection of pre-defined time steps, storing the time steps that require full sampling. For example, in a diffusion model, different time steps correspond to different stages in the evolution of an image from its initial state to its final state. Some time steps may require full sampling to obtain more randomness and diversity, or to better capture certain specific features of the image. By storing these time steps in the preset time step sequence, it is convenient to determine whether full sampling is needed in subsequent processes.

[0059] Specifically, if the current time step is within the preset time step sequence, the cached feature call condition is determined to be no cached feature call. In this case, the controller needs to perform complete sampling without relying on previous cached features. However, if the current time step is not within the preset time step sequence, the cached feature call condition is determined to be cached feature call. In this case, previously calculated and stored cached features can be utilized, avoiding redundant calculations, saving computational resources and time, and thus improving sampling efficiency.

[0060] In summary, this implementation method can flexibly choose whether to call cached features according to different time step requirements, thereby improving image inference efficiency.

[0061] S230. When the cache feature is invoked, the cache feature is obtained, and partial inference sampling is performed based on the initial noise and the cache feature to generate partial sampled features.

[0062] Specifically, when the cached feature call condition is met, the controller first retrieves the cached features, which are calculated and stored in the previous time step. Then, the controller combines the initial noise and the cached features to perform partial inference sampling. Partial inference sampling means sampling only a portion of the image, rather than sampling the entire image. Since the cached features already provide some information, only the uncertain parts need to be sampled.

[0063] S240. When the cached feature call status is "no cached feature call", perform full sampling based on the initial noise to generate full sampled features.

[0064] Specifically, if it is determined that cached features will not be invoked, the controller will perform full sampling based on the initial noise. Full sampling means sampling the entire image comprehensively, without relying on cached information.

[0065] Optionally, after performing full sampling based on the initial noise, the method further includes: determining intermediate features generated during the full sampling process; and storing the intermediate features to generate cached features.

[0066] During the complete sampling process based on the initial noise, the controller uses high-level features as intermediate features. Storing these intermediate features allows for their retrieval in subsequent sampling processes, improving sampling efficiency. By storing intermediate features, the same information is avoided from being recalculated in each sampling, thus saving computational resources and time.

[0067] Specifically, the generated cached features can be stored in various data structures. During subsequent sampling, when cached features need to be accessed, the corresponding features can be quickly retrieved based on specific indexes or query conditions.

[0068] S250. Generate the final image based on the current time step and sampling features.

[0069] Optionally, generating the final image based on the current time step and sampling features includes: determining whether the current time step has reached a preset time step threshold; if so, generating the final image based on the sampling features; otherwise, performing inference sampling for the next time step.

[0070] The technical solution of this invention generates initial noise by determining the target linear combination through a preset knowledge base, thereby skipping the early sampling step of the diffusion denoising process. By determining whether to perform partial inference sampling or full inference sampling based on different current time steps and initial noise, and by introducing cached features at some time steps to avoid performing full inference, only a small part of the inference is needed to achieve the effect of near-full inference, saving inference time and computational overhead, thereby achieving the purpose of accelerating the diffusion model.

[0071] Example 3

[0072] Figure 3 This is a schematic diagram of the structure of an image reasoning device provided in Embodiment 3 of the present invention. Figure 3 As shown, the device includes: an initial noise generation module 310, used to acquire an initial image, determine the current time step, and generate initial noise based on the initial image and the current time step;

[0073] The sampling feature generation module 320 is used to sample based on the current time step and initial noise to generate sampling features, wherein the sampling features include partial sampling features and complete sampling features;

[0074] The final image generation module 330 is used to generate the final image based on the current time step and sampling features.

[0075] Optionally, the initial noise generation module 310 specifically includes: an initial noise generation unit, used to: generate early noise based on the initial image when the current time step is 0; generate historical sampling features by performing several steps of inference sampling based on the early noise and the current time step; and when the current time step reaches a specified time step, match the historical sampling features through a preset knowledge base to generate initial noise.

[0076] Optionally, the initial noise generation unit specifically includes: a knowledge base matching subunit, used to: determine whether the preset knowledge base contains a specified number of features that are similar to the historical sampling features; if so, determine each linear combination of the specified number of features, take the linear combination with the highest similarity to the historical sampling features as the target linear combination, determine the target weight corresponding to the target linear combination, call the historical intermediate state of the trajectory of the target linear combination at a specified stage, and combine the historical intermediate state with the target weight to generate initial noise; otherwise, generate initial noise by random initialization based on the initial image.

[0077] Optionally, the sampling feature generation module 320 is specifically used to: determine the cache feature call status based on the current time step, wherein the cache feature call status includes calling the cache feature and not calling the cache feature; when the cache feature call status is calling the cache feature, obtain the cache feature, and perform partial inference sampling based on the initial noise and the cache feature to generate partial sampled features; when the cache feature call status is not calling the cache feature, perform full sampling based on the initial noise to generate full sampled features.

[0078] Optionally, the apparatus further includes: a cache feature generation module, used to determine intermediate features generated during the full sampling process after full sampling based on the initial noise; and to store the intermediate features to generate cache features.

[0079] Optionally, the sampling feature generation module 320 specifically includes: a cache feature call status determination unit, used to: obtain a preset time step sequence, wherein the preset time step sequence stores the time steps that need to be fully sampled; determine whether the current time step is in the preset time step sequence, if so, determine that the cache feature call status is not to call the cache feature; otherwise, determine that the cache feature call status is to call the cache feature.

[0080] Optionally, the final image generation module 330 is specifically used to: determine whether the current time step has reached the preset time step threshold; if so, generate the final image based on the sampling features; otherwise, perform inference sampling for the next time step.

[0081] The technical solution of this invention generates initial noise by determining the target linear combination through a preset knowledge base, thereby skipping the early sampling step of the diffusion denoising process. By determining whether to perform partial inference sampling or full inference sampling based on different current time steps and initial noise, and by introducing cached features at some time steps to avoid performing full inference, only a small part of the inference is needed to achieve the effect of near-full inference, saving inference time and computational overhead, thereby achieving the purpose of accelerating the diffusion model.

[0082] The image reasoning device provided in this embodiment of the invention can execute an image reasoning method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0083] Example 4

[0084] Figure 4A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0085] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0086] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0087] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as an image inference method.

[0088] In some embodiments, an image reasoning method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the image reasoning method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform an image reasoning method by any other suitable means (e.g., by means of firmware).

[0089] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0090] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0091] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0092] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0093] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0094] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0095] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0096] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. An image inference method applied to a diffusion model, characterized in that, include: Acquire an initial image and determine the current time step; then generate initial noise based on the initial image and the current time step. Sampling is performed based on the current time step and the initial noise to generate sampling features, wherein the sampling features include partial sampling features and complete sampling features; Generate the final image based on the current time step and the sampling features; The step of sampling based on the current time step and the initial noise to generate sampling features includes: The cache feature invocation status is determined based on the current time step, wherein the cache feature invocation status includes invoking the cache feature and not invoking the cache feature; When the cache feature is invoked, the cache feature is obtained, and partial inference sampling is performed based on the initial noise and the cache feature to generate partial sampled features; When the cached feature is not invoked, complete sampling is performed based on the initial noise to generate complete sampled features.

2. The method of claim 1, wherein, The step of generating initial noise based on the initial image and the current time step includes: When the current time step is 0, early noise is generated based on the initial image; Based on the early noise and the current time step, several steps of inference sampling are performed to generate historical sampling features. When the current time step reaches a specified time step, the historical sampling features are matched with a preset knowledge base to generate initial noise.

3. The method according to claim 2, characterized in that, The step of matching the historical sampling features with a preset knowledge base to generate initial noise includes: Determine whether the preset knowledge base contains a specified number of features that are similar to the historical sampling features. If so, determine each linear combination of the specified number of features, take the linear combination with the highest similarity to the historical sampling features as the target linear combination, determine the target weight corresponding to the target linear combination, call the historical intermediate state of the trajectory of the target linear combination at a specified stage, and combine the historical intermediate state with the target weight to generate initial noise. Otherwise, initial noise is generated randomly based on the initial image.

4. The method according to claim 1, characterized in that, After performing full sampling based on the initial noise, the method further includes: Identify the intermediate features generated during the complete sampling process; The intermediate features are stored to generate the cached features.

5. The method according to claim 1, characterized in that, The step of determining the cache feature call status based on the current time step includes: Obtain a preset time step sequence, wherein the preset time step sequence stores the time steps that need to be fully sampled; Determine whether the current time step is in the preset time step sequence; if so, determine that the cache feature call status is not to call the cache feature. Otherwise, the cache feature invocation situation is determined to be a cache feature invocation.

6. The method according to claim 1, characterized in that, The step of generating the final image based on the current time step and the sampled features includes: Determine whether the current time step has reached a preset time step threshold; if so, generate the final image based on the sampling features. Otherwise, proceed with inference sampling at the next time step.

7. An image reasoning device applied to a diffusion model, characterized in that, include: An initial noise generation module is used to acquire an initial image, determine the current time step, and generate initial noise based on the initial image and the current time step. A sampling feature generation module is used to sample based on the current time step and the initial noise to generate sampling features, wherein the sampling features include partial sampling features and complete sampling features; The final image generation module is used to generate a final image based on the current time step and the sampling features; Specifically, the sampling feature generation module is used to: determine the cache feature invocation status based on the current time step, wherein the cache feature invocation status includes invoking cache features and not invoking cache features; when the cache feature invocation status is invoking cache features, obtain cache features, and perform partial inference sampling based on the initial noise and the cache features to generate partial sampling features; when the cache feature invocation status is not invoking cache features, perform full sampling based on the initial noise to generate full sampling features.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.

9. A computer storage medium, characterized in that, The computer storage medium stores computer instructions that are used to cause a processor to execute the method of any one of claims 1-6.

Images