A low-latency image generation method based on latent consistency constraints

Abstract

The application discloses a low-delay image generation method based on latent consistency constraint and relates to the technical field of computer image processing. An image generation condition, a target image size, a latent variable space parameter and a preset inference position are read, an initial latent variable is formed, and a local track unit and a boundary transfer area are divided. Prediction clean latent variables, boundary transfer data and condition response data are obtained at each inference position, a local track offset unit is identified according to the local change, the boundary transfer relationship and the condition response state of adjacent inference positions, and a current effective reference is formed. According to the current effective reference, noise residuals and noise control parameters, local back-reference latent variables of the same noise position are formed, the boundary is closed by combining adjacent non-offset areas, and then the originally scheduled next inference position is entered. Finally, clean latent variables are obtained and a target image is generated by decoding. The application can inhibit the propagation of local track offset without increasing the number of local independent inferences.

Description

Technical Field

[0001] This invention relates to the field of computer image processing technology, specifically to a low-latency image generation method based on potential consistency constraints. Background Technology

[0002] In existing image generation techniques, diffusion models typically obtain the target image step by step through multiple denoising processes. This generation process requires repeated calls to the generator network, resulting in relatively long inference times. To reduce the number of inference steps, latent consistency models map the latent variable states at different noise stages, enabling image generation to be completed within fewer inference steps, thereby shortening image generation time.

[0003] With fewer inference steps, the latent variable states that need to be completed between adjacent inference positions can change significantly. For images containing small objects, text, occluded edges, and areas where multiple objects intersect, the generated states at different spatial locations are not entirely consistent. Some areas may already show structural shifts, while the overall state of the latent variables as a whole may not yet show obvious anomalies. Due to the limited number of subsequent inference steps, such local shifts are difficult to correct themselves by the subsequent generation process and may continue to affect adjacent areas, ultimately leading to local structural discontinuities, abnormal edge connections, and inconsistencies between the generated object attributes and the input conditions.

[0004] Existing processing methods can increase the number of sampling steps to give the offset regions more opportunities for correction, or they can re-call the generative network to process the relevant content. However, both of these methods increase the number of model inferences and the amount of computation, which conflicts with the goal of reducing processing time through fewer-step image generation. Therefore, it is necessary to identify and process the latent variable regions that have already experienced local offsets during the generation process without changing the original fewer-step inference process, and to limit the impact of these local offsets on subsequent generation results. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a low-latency image generation method based on potential consistency constraints, thereby solving the problems mentioned in the background section.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a low-latency image generation method based on potential consistency constraints, comprising the following steps: S1. Read the image generation conditions, target image size, latent variable space parameters corresponding to the latent consistency image generation model, and preset inference position to form initial latent variables, and divide the local trajectory units and the boundary junction areas between adjacent local trajectory units according to the fixed positions in the latent variable space. S2. Generate the entire latent variable according to the preset inference position, obtain the predicted clean latent variable and conditional response data at each inference position, and extract the corresponding predicted clean latent variable fragment, boundary junction data and conditional response fragment according to the fixed position of the local trajectory unit. S3. Based on the changes in the predicted clean latent variables of the same local trajectory unit in adjacent inference positions, the boundary intersection relationship of the corresponding directions of adjacent local trajectory units, and the main condition response identifier and condition response change amount of the corresponding image generation condition constraints, register the local trajectory offset unit, and register the predicted clean latent variable segments that have not been registered as local trajectory offsets but satisfy the local change boundary, boundary intersection boundary and condition response boundary as the current valid benchmark. S4. For local trajectory offset units, read the last formed current effective reference before the current inference position, the local noise residual data corresponding to the current inference position, and the noise control parameters. Based on the noise control parameters, map the latent variable structure components of the current effective reference to the current noise state, and combine them with the local noise residual data to form local back-reference latent variables at the same noise position. Then, based on the current latent variables of adjacent non-offset local trajectory units, perform closure on the boundary intersection area determined by S1. S5. After completing the local backreference and boundary closure, the entire latent variable is sent to the next preset inference position. The remaining preset inference positions are processed according to S2 to S4. After obtaining the final clean latent variable at the last preset inference position, the latent variable decoding is performed.

[0007] Furthermore, S1 includes: The latent variable spatial size is determined based on the target image size and the spatial scaling ratio corresponding to the latent consistency image generation model. Initial latent variables with the same latent variable spatial size and number of latent variable channels are formed based on the random noise seed corresponding to the current image generation task. Local trajectory units are divided according to fixed coordinates in the latent variable space, and corresponding boundary junction regions are established between local trajectory units that have vertical and horizontal adjacent relationships. The fixed coordinates of the local trajectory unit and the position of the boundary intersection region remain unchanged across all preset inference positions in the current image generation task.

[0008] Furthermore, S2 includes: After completing the generation of the entire latent variable at each preset inference position, the current predicted clean latent variable is saved in correspondence with the current inference position identifier; Based on the fixed coordinates of the local trajectory unit, extract the corresponding predicted clean latent variable fragments and boundary junction data from the current predicted clean latent variables, and extract the corresponding condition response fragments from the condition response data at the current inference position. Establish a correspondence between local trajectory unit numbers, current inference position identifiers, predicted clean latent variable segments, boundary handover data, and conditional response segments.

[0009] Furthermore, S3 includes: Read the predicted clean latent variable fragments corresponding to the same fixed spatial coordinates in the previous and current inference positions according to the local trajectory unit number; The changes between two predicted clean latent variable segments are determined based on the local change boundary corresponding to the current inference position; When the predicted changes in the cleaning latent variable exceed the local change boundary corresponding to the current inference position, a local change anomaly flag is registered for the corresponding local trajectory unit.

[0010] Furthermore, S3 also includes: For local trajectory units with direct spatial connections, boundary intersection amounts are formed according to the corresponding connection directions. When the boundary intersection amount formed in the corresponding connection direction exceeds the boundary intersection boundary corresponding to the current inference position, a boundary mismatch flag is registered in the corresponding direction. Compare the primary condition response identifiers corresponding to the same local trajectory unit in the previous inference position and the current inference position. If the primary condition response identifiers corresponding to the two inference positions are inconsistent, register the condition migration flag. If the primary condition response identifiers corresponding to the two inference positions are consistent, form the corresponding condition response change amount. Register the condition migration flag when the condition response change amount exceeds the condition response boundary corresponding to the current inference position. When the local change anomaly flag and the boundary mismatch flag are both set, the local trajectory offset unit is registered; when the local change anomaly flag and the condition migration flag are both set, the local trajectory offset unit is registered. The predicted clean latent variable fragments that are not registered local trajectory offset units, whose local changes are within the corresponding local change boundary, whose boundary intersection amounts formed by each corresponding connection direction are within the corresponding boundary intersection boundary, whose main conditional response identifiers corresponding to the previous inference position and the current inference position are consistent, and whose conditional response changes are within the corresponding conditional response boundary, are registered as the current valid benchmarks.

[0011] Furthermore, S4 includes: Retrieve the currently valid reference formed before the current inference position according to the local trajectory unit number of the local trajectory offset unit, and select the last currently valid reference formed before the current inference position according to the preset inference position order; Read the local noise residual data corresponding to the current inference position according to the fixed spatial coordinates of the local trajectory offset unit, and read the noise control parameters corresponding to the current inference position; Using the current valid benchmark as the source of the clean structure and the local noise residual data as the noise source of the current generation task, a local back-reference latent variable is formed that is at the same noise position as the current whole latent variable.

[0012] Furthermore, S4 also includes: The local trajectory offset unit is divided into an internal recovery region and a boundary junction region located outside the internal recovery region. The local back-reference latent variable is written into the internal recovery region. Read the current latent variables of adjacent unoffset local trajectory units that have a direct spatial connection with the local trajectory offset unit; Based on the continuous configuration of the boundary, determine the combined weights corresponding to the edge data of the local latent variable and the edge data of the current latent variable of the adjacent non-offset local trajectory unit. Perform closure on the boundary junction area according to the combined weights, and write the local latent variable after closure back to the fixed spatial position corresponding to the current whole latent variable.

[0013] Furthermore, S5 includes: The entire latent variable after local backreference and boundary closure is used as the input for the next preset inference position, while keeping the current image generation conditions, local trajectory unit fixed coordinates and preset inference position order unchanged; At the next preset inference position, the predicted clean latent variables, boundary junction data and condition response data are obtained again, and the local trajectory offset judgment is re-executed; When the local trajectory unit after local backtracking has a local change within the corresponding local change boundary at the next preset inference position, the boundary intersection amount formed by each corresponding connection direction is within the corresponding boundary intersection boundary, the main condition response identifiers corresponding to the previous inference position and the next preset inference position are consistent, and the condition response change amount is within the corresponding condition response boundary, the corresponding local trajectory offset unit identifier is removed, and the predicted clean latent variable segment corresponding to the next preset inference position is updated to the current valid benchmark.

[0014] Furthermore, S5 also includes: After obtaining the final clean latent variable at the last preset inference position, the current valid reference formed last in the preset inference position order is read for the fixed spatial position that is still registered as a local trajectory offset unit before the last preset inference position. When the boundary intersection amount formed by the boundary intersection data corresponding to the current effective benchmark and the final cleaning latent variable data of the adjacent unoffset local trajectory unit corresponding to the last preset inference position is within the corresponding boundary intersection boundary range, and the main condition response identifier corresponding to the current effective benchmark is consistent with the main condition response identifier corresponding to the last preset inference position, and the condition response change amount formed by the corresponding condition response values ​​is within the corresponding condition response boundary range, the internal region of the corresponding local trajectory offset unit in the final cleaning latent variable is restored with the current effective benchmark, and the boundary intersection region is closed according to the final cleaning latent variable data corresponding to the adjacent unoffset local trajectory unit. Perform latent variable decoding on the final cleaned latent variables after local processing.

[0015] Compared with existing technologies, this invention provides a low-latency image generation method based on potential consistency constraints, which has the following advantages: 1. By reading the image generation conditions, target image size, latent variable spatial parameters, and preset inference positions, initial latent variables are formed, and local trajectory units and boundary junction regions at fixed positions are divided. Predicted clean latent variables, boundary junction data, and conditional response data are extracted at each inference position. Local trajectory offsets are identified based on the changes in latent variables, boundary junction relationships, and conditional response states of the same local trajectory unit in adjacent inference positions. Local back-reference latent variables at the same noise position are formed using the current effective benchmark, current noise residue, and noise control parameters. Local back-reference and boundary closure are performed on the offset region, so that local latent variables can be recovered within the original few-step generation process after deviating from the effective generated trajectory. This reduces the continuous propagation of local structural breaks, edge adhesion, and object attribute mismatches in subsequent inference positions, and solves the technical problem that in the generation of minimally invasive latent consistency images, the overall state is not abnormal but the local trajectory has already shifted, and the remaining inference positions are insufficient for natural recovery.

[0016] 2. By continuing to send the entire latent variable after local backtracking to the original next preset inference position, and keeping the image generation conditions, the fixed position of the local trajectory unit and the preset inference order unchanged, the normal area and the area that has completed local restoration continue to be in the same latent variable generation process; at the same time, the boundary of the backtracking area is closed by using adjacent unoffset local trajectory units, and the latent variable decoding is performed uniformly after obtaining the final clean latent variable at the last preset inference position, which can reduce the spatial seams and discontinuities in the generation state caused by local restoration. At the same time, there is no need to add an independent complete generation network inference process for the local offset area, which is conducive to maintaining the low latency characteristics of latent consistency image generation. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the process of a low-latency image generation method based on potential consistency constraints according to the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Example: Figure 1 A low-latency image generation method based on latent consistency constraints is presented, which is performed according to the following steps: S1. Read the image generation conditions, target image size, latent variable space parameters corresponding to the latent consistency image generation model, and preset inference position to form initial latent variables, and divide the local trajectory units and the boundary junction areas between adjacent local trajectory units according to the fixed positions in the latent variable space. S2. Generate the entire latent variable according to the preset inference position, obtain the predicted clean latent variable and conditional response data at each inference position, and extract the corresponding predicted clean latent variable fragment, boundary junction data and conditional response fragment according to the fixed position of the local trajectory unit. S3. Based on the changes in the predicted clean latent variables of the same local trajectory unit in adjacent inference positions, the boundary intersection relationship of the corresponding directions of adjacent local trajectory units, and the main condition response identifier and condition response change amount of the corresponding image generation condition constraints, register the local trajectory offset unit, and register the predicted clean latent variable segments that have not been registered as local trajectory offsets but satisfy the local change boundary, boundary intersection boundary and condition response boundary as the current valid benchmark. S4. For local trajectory offset units, read the last formed current effective reference before the current inference position, the local noise residual data corresponding to the current inference position, and the noise control parameters. Based on the noise control parameters, map the latent variable structure components of the current effective reference to the current noise state, and combine them with the local noise residual data to form local back-reference latent variables at the same noise position. Then, based on the current latent variables of adjacent non-offset local trajectory units, perform closure on the boundary intersection area determined by S1. S5. After completing the local backreference and boundary closure, the entire latent variable is sent to the next preset inference position. The remaining preset inference positions are processed according to S2 to S4. After obtaining the final clean latent variable at the last preset inference position, the latent variable decoding is performed.

[0020] S1 specifically refers to: After receiving an image generation request, the image generation device reads the image generation conditions, target image size, latent consistency image generation model file, and sampling configuration data corresponding to the model file for this image generation. The image generation conditions define the object content, object attribute relationships, and spatial layout relationships for the generated image; the target image size defines the pixel size of the final output image; the latent consistency image generation model file provides the generation process in the latent variable space; and the sampling configuration data provides the preset inference positions to be executed sequentially during this image generation process.

[0021] The image generation conditions are provided by the current image generation task and their content remains unchanged during this generation process. Upon receiving these conditions, they are mapped to the latent consistency image generation model file being invoked, ensuring that subsequent pre-defined inference positions are updated with latent variables under the same constraints. The target image size is read from the current generation task configuration file. The recorded pixel width and pixel height, combined with the spatial scaling configuration provided in the model file, are used to determine the coordinate correspondence between the output image space and the latent variable space.

[0022] The current model file's configuration data records the latent variable space size, the number of latent variable channels, the scaling ratio of the latent variable space relative to the output image space, and the number of preset inference positions. This data serves as the latent variable space parameters used in this generation. After completing the model configuration read, initial latent variables are established based on the random noise seed configured for this image generation task. This random noise seed corresponds only to the current generation task and remains unchanged throughout the entire process from the formation of the initial latent variables to the end of the final generation.

[0023] The initial latent variables are established according to the spatial size, number of channels, and data arrangement specified by the current model. Taking a target output image of 1024 pixels by 1024 pixels and a model spatial scaling ratio of eight as an example, the corresponding latent variable spatial size is 128 by 128, and the initial data is written to each position according to the number of latent variable channels specified by the model. If the target output size changes, the latent variable space and its positional correspondence are re-determined according to the output size and the current model spatial scaling configuration. The latent variable spatial positions formed by other output sizes in the past are not used as the basis for the positions generated this time.

[0024] After the initial latent variables are formed, the image generation device divides the space into local trajectory units based on fixed coordinates in the latent variable space. These local trajectory units define the comparison range for latent variable changes between different preset inference positions within the same spatial region. The positions of the local trajectory units are determined before the current image generation task begins and remain fixed throughout the entire image generation process, without being readjusted based on object changes, texture changes, or edge changes occurring in the current generation stage.

[0025] The size of the local trajectory unit is determined by combining the latent variable space size of the current latent consistency image generation model and the deployment validation results. During model deployment, validation samples are set according to the output size of the current model and the latent variable scaling ratio, and small objects, object boundaries, text regions, slender structures, and multi-object intersections are selected as observation objects. The generation results corresponding to different local trajectory unit sizes are compared, and the structural changes of the same spatial region in continuous inference positions are recorded. Based on this, the local trajectory unit size suitable for the current model is determined, and the determined size is written to the corresponding model configuration file.

[0026] In specific implementation, when the output image size is 1024 pixels by 1024 pixels and the latent variable space size is 128 pixels by 128 pixels, the latent variable space is divided into multiple local trajectory units at fixed positions. Each local trajectory unit covers eight by eight latent variable positions, forming a 16x16 local trajectory unit arrangement. This size is only used as the implementation method corresponding to the current model configuration and is not a fixed limitation for all models. When the output image size, latent variable space size, or model structure changes, the local trajectory unit size is re-determined based on the corresponding model deployment and verification results.

[0027] After completing the local trajectory unit division, the image generation device further establishes the boundary junction regions between adjacent local trajectory units. These boundary junction regions are used to preserve the continuous change relationships between spatially adjacent local trajectory units, and their positions are determined based on the fixed spatial coordinates between the local trajectory units. For local trajectory units with vertical adjacency, corresponding vertical boundary junction regions are established; for local trajectory units with horizontal adjacency, corresponding horizontal boundary junction regions are established.

[0028] The boundary junction region is taken from the intersection of adjacent local trajectory units, and its data consists of latent variable data from both sides near the connecting edge. This region serves only as the observation range for the continuous relationship between adjacent local trajectory units; it does not change the spatial division of the original local trajectory units, nor does it form a new generated region. The width of the boundary junction region is determined during model deployment and validation and recorded in the corresponding local trajectory configuration.

[0029] During deployment and verification, normally generated samples are selected, and the changes in latent variables of adjacent structural regions at each preset inference position are recorded. The continuity performance of adjacent regions under different boundary widths is compared, and the boundary junction width that can reflect the changes in the direct connection position is determined accordingly. In this embodiment, the width of a latent variable position is used as the boundary junction area to obtain data at the connection point of adjacent local trajectory units, while limiting the local processing range to the vicinity of the junction position.

[0030] After establishing the local trajectory units and boundary junction regions, the image generation device reads the preset inference positions from the sampling configuration data and establishes the subsequent latent variable generation order according to the preset inference positions. The preset inference positions are derived from the sampling configuration file corresponding to the current latent consistency image generation model and are not dynamically added or removed during the current image generation process. The initial latent variables, local trajectory unit positions, boundary junction region positions, and preset inference positions together form the basic data for latent variable processing corresponding to the current image generation task.

[0031] If the read model configuration data is incomplete, latent variable spatial parameters are missing, or the current target image size exceeds the model's supported range, the current generation will not continue to establish initial latent variables, but will return to the corresponding model configuration for re-reading. If the random noise seed does not correspond to the current latent variable spatial size, the current initialization process will be terminated before the local trajectory unit division to avoid using data with inconsistent spatial dimensions for subsequent comparisons.

[0032] After completing the above processing, the image generation conditions, initial latent variables, local trajectory units, boundary junction regions, and preset inference positions required for this image generation have all been determined. All data are stored using the same latent variable spatial coordinate relationship, and subsequent local data at different inference positions are extracted and compared accordingly, thereby ensuring that the same local trajectory unit has a consistent spatial correspondence between consecutive inference positions.

[0033] S2 specifically refers to: After initializing S1, the generation process begins according to the preset inference positions registered in the sampling configuration file. Each preset inference position sequentially calls the current latent consistency image generation model, always using the entire latent variable as the model input. The local trajectory units defined by S1 serve only as fixed spatial ranges for subsequent data extraction and comparison, and are not individually fed into the model for generation. Thus, each local trajectory unit remains within the same generation task, the same image generation conditions, and the continuous noise state changes, and the subsequently obtained local data share a common latent variable spatial basis.

[0034] After each preset inference position is generated, the corresponding clean latent variable is obtained from the model output. This latent variable records the model's prediction of the clean latent variable state at the current stage and remains in latent variable form, without being converted into a pixel image. Subsequent processing of local trajectory units directly uses this model output, so this step does not require additional image decoding or image recognition results from outside the model.

[0035] When obtaining the predicted clean latent variables, the inference position of this processing is also recorded. The inference position identifier is determined according to the preset sequence number in the sampling configuration file and is written together with the corresponding predicted clean latent variables into the data buffer of the current image generation task. When reading data from adjacent inference positions, this identifier is used to determine the order of each group of data, so that the same local trajectory unit can establish a correspondence between the previous inference position and the current inference position, avoiding the mixing of data from different inference positions.

[0036] After the current inference position is generated, conditional response data is also obtained from the output data of the image generation conditional interactions within the model. This data reflects the response of the current image generation condition at a fixed position in the latent variable space, and its spatial position corresponds to the subsequent data extraction of local trajectory units. Here, the conditional interaction results already generated within the model during this generation process are used; no separate conditional analysis model is set up, nor is an independent conditional recognition process added.

[0037] When reading the conditional response data, the same spatial coordinates as the predicted clean latent variables are used, and the data is kept consistent with the fixed coordinates of the local trajectory units determined in S1. This ensures that the conditional response data read from the same local trajectory unit at different inference positions are consistent.

[0038] After completing the reading of the predicted cleaning latent variables and conditional response data for the current inference location, the image generation device performs spatial extraction on the data for the current inference location according to the local trajectory unit division rules fixed in S1. The extraction process reads data only according to fixed spatial coordinates, without changing the original spatial arrangement of the predicted cleaning latent variables or the original spatial arrangement of the conditional response data. For each local trajectory unit, the image generation device reads the predicted cleaning latent variable data within the coverage area of ​​that local trajectory unit and registers this data as the predicted cleaning latent variable fragment for the corresponding local trajectory unit at the current inference location.

[0039] While extracting the predicted cleaning latent variable data within local trajectory units, the image generation device directly reads the boundary junction region data around the local trajectory unit from the predicted cleaning latent variable data at the same current inference position. The position of the boundary junction region is consistent with the boundary junction region determined in S1 and is not redefined in the current step. For local trajectory units in the middle position, the boundary junction region data of their upper, lower, left, and right sides are read respectively; for local trajectory units located at the edge of the entire latent variable, only the actual boundary junction region data is read, and no non-existent outer region is constructed. The boundary junction data obtained in this way is in the same spatial scope and the same inference position scope as the predicted cleaning latent variable segment at the current inference position, providing a unified source for subsequent determination of the boundary junction relationship between adjacent local trajectory units.

[0040] The image generation device also extracts the conditional response fragments of the corresponding local trajectory units from the conditional response data at the current inference position, using the same fixed spatial coordinates as the local trajectory units. Each conditional response fragment is bound to a local trajectory unit number and written to a buffer along with the current inference position identifier. This allows the image generation device to perform corresponding judgments on the main conditional response identifiers and corresponding conditional response values ​​of the same local trajectory unit in both preceding and following inference positions, without mixing conditional response data from different spatial positions.

[0041] In this embodiment, the latent variable space is set to 128 x 128, with each local trajectory unit corresponding to eight x eight latent variable positions. After dividing according to the aforementioned fixed coordinates, the corresponding predicted clean latent variable segments are obtained from the coverage area of ​​each local trajectory unit, and the data of the boundary junction area around its perimeter is simultaneously retained. Taking local trajectory unit A-05 as an example, at the first inference position, the predicted clean latent variable segments, the upper, lower, left, and right boundary junction data, and the corresponding conditional response segments within the fixed coordinate range of this unit are saved, and the first inference position identifier is recorded. After entering the second inference position, the corresponding data is still obtained for the same fixed coordinate range of A-05, and the second inference position identifier is recorded. The third and fourth inference positions use the same data extraction range and registration method. Thus, the data of the same local trajectory unit in different inference positions can be matched by the local trajectory unit number, fixed spatial coordinates, boundary junction area position, and inference position identifier.

[0042] After extracting data from all local trajectory units at the current inference position, the image generation device writes the corresponding predictive clean latent variable fragment, boundary intersection data, conditional response fragment, and current inference position identifier for each local trajectory unit into the data buffer of the same image generation task. The buffer establishes a retrieval relationship based on the local trajectory unit number and the inference position identifier, enabling subsequent steps (S3) to directly read the corresponding data of a specific local trajectory unit at adjacent inference positions. The cached data is used only in the current image generation task, is not accessed across tasks, and is not shared with other image generation tasks.

[0043] When a preset inference location completes the generation process but fails to obtain complete predicted cleaning latent variables, the image generation device marks this inference location as a data incomplete location and stops extracting local trajectory unit data for that location; subsequently, this inference location will not be used in local trajectory offset judgment. If the conditional response data for the current inference location is missing, the image generation device can still save the predicted cleaning latent variable fragments and boundary junction data, but it will write a conditional response missing marker to the corresponding local trajectory unit. In subsequent step S3, the consistency judgment of the main conditional response identifier and the judgment of the change in conditional response will not be performed based on this conditional response data.

[0044] When the boundary junction data is incomplete, the image generation device only saves the predicted clean latent variable fragment and conditional response fragment within the local trajectory unit, and writes a boundary junction data missing mark on the local trajectory unit. This set of data will not be used as the direct basis for forming the current effective benchmark.

[0045] Through the above process, the image generation device generates a data set corresponding one-to-one with a fixed local trajectory unit at each preset inference position. This data set corresponds at least to the current inference position identifier, local trajectory unit number, predicted clean latent variable fragment, boundary junction data, and conditional response fragment. All data originates from the actual generation output of the current image generation task at the current inference position and maintains a consistent spatial coordinate and inference position caliber, providing a direct data foundation for subsequent judgments of local trajectory offsets based on data changes of the same local trajectory unit between adjacent inference positions.

[0046] S3 specifically refers to: After completing data acquisition at two adjacent preset inference positions, the image generation device reads the data sets corresponding to the previous and current inference positions from the data buffer of the current image generation task, and establishes a one-to-one correspondence according to the local trajectory unit number. This correspondence is formed only based on the fixed spatial coordinates and local trajectory unit numbers determined in S1, and does not re-divide the spatial region based on the object outline, edge position, or texture changes in the currently generated image, so that the same local trajectory unit always corresponds to the same latent variable spatial range between different inference positions.

[0047] The image generation device first reads the predicted clean latent variable fragments corresponding to the same local trajectory unit in the previous and current inference positions. The predicted clean latent variable fragments in both inference positions are derived from the output of the latent consistency image generation model after completing the entire latent variable generation process at the corresponding inference position, and have already been extracted by S2 according to the same fixed spatial coordinates.

[0048] When comparing, the image generation device only compares the predicted cleaning latent variable data at the same spatial coordinates within the same local trajectory unit. The change in the predicted cleaning latent variable is formed based on the difference in the predicted cleaning latent variable segments at the corresponding spatial coordinates of the same local trajectory unit. The difference is summarized according to the following fixed caliber to form the corresponding change in the predicted cleaning latent variable, and is compared with the local change boundary corresponding to the current inference position. The data difference between the current local trajectory unit and other local trajectory units is not used as the basis for local change.

[0049] For the same local trajectory unit, a one-to-one correspondence is established between the predicted clean latent variable data of the previous inference position and the current inference position according to fixed spatial coordinates and latent variable channels. The absolute difference is calculated for each channel data of each corresponding latent variable position. The absolute differences of all corresponding positions within the coverage area of ​​the local trajectory unit are then summarized, and the arithmetic mean of all absolute differences is used as the change in predicted clean latent variable for that local trajectory unit between the previous and current inference positions. The same comparison object and summarization caliber are used when forming local change boundaries during the model deployment and validation phase, and this caliber remains unchanged during the operational phase.

[0050] Local change determination uses local change boundaries bounded to the current inference position. These local change boundaries are derived from the deployment and verification process of the current potential consistency image generation model.

[0051] In this embodiment, the local change boundaries, boundary junction boundaries, and conditional response boundaries all originate from the unified verification process during the deployment phase of the current latent consistency image generation model. During the deployment verification phase, the model file, output image size, latent variable spatial parameters, and preset inference positions are fixed. Normal generation samples corresponding to the actual generation task are selected to execute the complete inference process. Changes in the predicted clean latent variables, boundary junction regions, and conditional responses at each preset inference position are recorded according to the local trajectory unit number. A normal change range is formed based on the stable change records of the same local trajectory unit in continuous inference positions, and corresponding boundary parameters are generated according to fixed statistical rules. Subsequently, the formed local change boundaries, boundary junction boundaries, and conditional response boundaries are associated and saved with the corresponding model version number, model configuration number, sampling configuration number, and local trajectory unit number for subsequent local trajectory offset judgment.

[0052] The boundary parameters mentioned above were obtained by performing verification using fixed model files, fixed sampling configurations, and fixed verification samples before the model went online. They were also associated with and saved with the corresponding model version number to ensure that the boundary parameters generated by the corresponding verification were called when different model versions were running.

[0053] During the deployment and validation process, the model file, output image size, latent variable spatial parameters, and preset number of inference positions are fixed. Normal generated samples containing human edges, object boundaries, text regions, slender structures, and complex texture regions are selected. The normal variation range of the predicted clean latent variables within the same local trajectory unit between adjacent inference positions is collected. According to the fixed statistical caliber used in the current model deployment and validation, the normal variation range corresponding to each inference position is formed into a local variation boundary and written into the local trajectory configuration file.

[0054] During operation, the image generation device reads the local change boundary corresponding to the current inference position and compares the predicted cleaning latent variable change of the same local trajectory unit between the previous and current inference positions with the local change boundary corresponding to the current inference position. When the predicted cleaning latent variable change within the local trajectory unit is within the range of the local change boundary corresponding to the current inference position, the current local trajectory unit is registered as a normal local change state; when the predicted cleaning latent variable change within the local trajectory unit exceeds the local change boundary corresponding to the current inference position, the current local trajectory unit is registered as an abnormal local change state, and a local change abnormality flag is written.

[0055] After a local change state is established, the image generation device further reads the boundary intersection data corresponding to the local trajectory unit at the current inference position, and simultaneously reads the boundary intersection data of the local trajectory units directly adjacent to the local trajectory unit in the corresponding direction. All boundary intersection data originates from the data extracted by S2 from the predicted clean latent variables at the current inference position; no new boundary data is generated separately.

[0056] For each direction with a direct spatial connection, the image generation device establishes a one-to-one correspondence between the boundary intersection data of the current local trajectory unit near the connection direction and the boundary intersection data of the corresponding adjacent local trajectory units near the same connection position, according to the fixed spatial arrangement position and latent variable channels in the boundary intersection region. It generates an absolute difference for the channel data at each corresponding position and uses the arithmetic mean of the absolute differences of all corresponding positions in the boundary intersection region of that connection direction as the boundary intersection quantity for that direction. The upper, lower, left, and right sides each generate their own boundary intersection quantities, without merging data from different connection directions to form the same boundary intersection quantity.

[0057] The image generation device only performs boundary intersection judgment on directions with direct spatial connection relationships, that is, it judges the directions with adjacent local trajectory units on the upper, lower, left and right sides respectively, and does not compare the data relationship between non-adjacent directions or distant regions.

[0058] Boundary intersection determination uses the boundary intersection boundary bound to the current inference position. The boundary intersection boundary originates from the boundary change range of adjacent local trajectory units under normal continuous generation conditions during deployment and verification. During verification, normally generated samples containing continuous edge structures, object contour connection structures, and text stroke connection structures are used to collect boundary intersection data of adjacent local trajectory units at the same inference position. According to the fixed statistical caliber used in the current model deployment and verification, the normal boundary change range corresponding to each inference position is formed into the boundary intersection boundary and written into the local trajectory configuration file.

[0059] During operation, the image generation device performs boundary intersection judgment for each direction where the current local trajectory unit has adjacent units. When the boundary intersection amount formed by the current local trajectory unit and its corresponding adjacent local trajectory unit in a certain direction exceeds the boundary intersection boundary corresponding to the current inference position, a boundary mismatch flag is registered in that direction; when the boundary intersection amount formed in that direction is within the boundary intersection boundary range corresponding to the current inference position, a boundary continuity state is registered in that direction. For directions located at the edge of the entire latent variable and outside which there are no adjacent local trajectory units, no boundary intersection judgment is performed, and no boundary mismatch flag is registered.

[0060] When conditional response data is available during the current image generation process, conditional response segments corresponding to the same fixed spatial range between the previous and current inference positions are read according to the local trajectory unit number. All read conditional response segments are taken from the output results of the conditional interactions within the latent consistency image generation model. Subsequent comparisons directly use the data from within this model, without introducing additional external image recognition results.

[0061] The image generation device reads the conditional response segments according to the same fixed spatial coordinates as the local trajectory units, and identifies the main conditional response identifiers involved in the current image generation conditional constraints within the current local trajectory unit. These main conditional response identifiers are derived from existing data in the conditional response segments within the current local trajectory unit, eliminating the need to rebuild a new recognition model.

[0062] For the same local trajectory unit, the primary conditional response identifier is determined from the conditional identifiers that have already participated in the conditional interaction of the potential consistency image generation model in the current image generation conditions. The image generation device reads the response values ​​corresponding to all fixed spatial positions within the coverage area of ​​each conditional identifier according to the fixed spatial coordinates of the local trajectory unit, and calculates the arithmetic mean of all response values ​​corresponding to the same conditional identifier. This arithmetic mean is used as the conditional response value of the corresponding conditional identifier in the current local trajectory unit. The conditional response values ​​corresponding to each conditional identifier are sorted, and the conditional identifier with the largest conditional response value is registered as the primary conditional response identifier for the current local trajectory unit. When two or more conditional identifiers have the same conditional response value, the conditional identifier registered earlier in the fixed registration order of the conditional identifiers in the current image generation conditions is selected as the primary conditional response identifier. All preset inference positions for the same image generation task maintain the same set of conditional identifiers, fixed spatial range, and calculation method for conditional response values.

[0063] For the same local trajectory unit in the previous and current inference positions, the primary conditional response identifiers determined according to the above criteria are first compared. If the primary conditional response identifier changes, the change is recorded as an identifier migration change. If the primary conditional response identifier remains consistent, the conditional response values ​​corresponding to the primary conditional response identifier in the previous and current inference positions are read, and the absolute difference between the two conditional response values ​​is calculated. This absolute difference is then used as the corresponding conditional response change. When forming conditional response boundaries during the model deployment and verification phase, the same rules for determining primary conditional response identifiers, the same criteria for calculating conditional response values, and the same criteria for forming conditional response changes are used.

[0064] Conditional response judgment uses conditional response boundaries bound to the current inference position. These conditional response boundaries are derived from the model deployment and validation process. During validation, conditional categories are generated for a fixed image, and the normal conditional response variation range of the same local trajectory unit at different inference positions is collected. Conditional response boundaries for the corresponding inference positions are formed according to the fixed statistical criteria used in the current model deployment and validation, and written to the local trajectory configuration file.

[0065] During operation, the image generation device first compares the principal conditional response identifiers corresponding to the same local trajectory unit in the previous inference position with those in the current inference position. When the principal conditional response identifiers corresponding to the two inference positions are inconsistent, the current local trajectory unit is directly registered as a conditional transition state, and a conditional transition flag is written. When the principal conditional response identifiers corresponding to the two inference positions are consistent, the conditional response values ​​corresponding to the principal conditional response identifiers in the two inference positions are read, and a conditional response change is formed according to the aforementioned rules. This conditional response change is then compared with the conditional response boundary corresponding to the current inference position. When the conditional response change is within the conditional response boundary range, the current local trajectory unit is registered as a conditional response hold state. When the conditional response change exceeds the conditional response boundary, the current local trajectory unit is registered as a conditional transition state, and a conditional transition flag is written.

[0066] The image generation device does not directly register local trajectory offset units based on a single state. Instead, it determines the status based on a combination of local change states, boundary transition states, and conditional response states. For the same local trajectory unit, when the local change anomaly flag is set and a boundary mismatch flag has been set for any direction with adjacent local trajectory units, the local trajectory unit is registered as a local trajectory offset unit. For the same local trajectory unit, when both the local change anomaly flag and the conditional migration flag are set, the local trajectory unit is registered as a local trajectory offset unit even if a boundary mismatch has not yet formed in the boundary transition relationships of each direction.

[0067] If only local change anomaly indicators exist but boundary mismatch indicators and condition migration indicators are absent, the unit will not be registered as a local trajectory offset unit and will remain under observation. When boundary junction data is missing, local trajectory offset judgment will not be based on boundary junction relationships. When condition response data is missing, consistency judgment of the main condition response identifier and judgment of condition response change amount will not be performed based on the condition response data, but registration of local trajectory offset units based on the combination relationship between local change anomaly indicators and boundary mismatch indicators is still permitted.

[0068] After completing the registration of local trajectory offset units, the image generation device performs a current valid reference formation judgment on local trajectory units that are not registered as local trajectory offset units. The current valid reference comes from the predicted clean latent variable fragments that have been generated and passed the trajectory judgment at the current inference position in the current image generation task. It is not a pre-saved fixed image template, nor is it extracted from other image generation tasks.

[0069] For a local trajectory unit that has not been registered as a local trajectory offset unit, the image generation device reads the predicted clean latent variable fragment, boundary junction data in each direction, and conditional response fragment corresponding to the current inference position, and checks the current local change state, boundary junction state, and conditional response state in sequence.

[0070] When the local change of the local trajectory unit is within the local change boundary range corresponding to the current inference position, and the boundary intersection amounts formed by the directions of all adjacent local trajectory units are within the boundary intersection boundary range corresponding to the current inference position, and the main conditional response identifiers corresponding to the previous inference position and the current inference position are consistent, and the corresponding conditional response change amounts are within the conditional response boundary range corresponding to the current inference position, the predicted clean latent variable fragment of the current inference position is registered as the current effective benchmark of the local trajectory unit.

[0071] When registering a current valid benchmark, the following information is simultaneously saved: local trajectory unit number, inference location identifier forming the current valid benchmark, corresponding predicted clean latent variable segment, boundary intersection data in each direction, main conditional response identifier, and the conditional response value corresponding to the main conditional response identifier. All of the above data is written to the data cache of the current image generation task and is used only in the current image generation task, without being accessed across tasks.

[0072] When a local trajectory unit is not registered as a local trajectory offset unit, but any one of its local changes, boundary intersection state, or conditional response state has not yet met the conditions for forming a current valid benchmark, it is not formed as a current valid benchmark. Instead, the local trajectory unit is registered as a region to be observed. The region to be observed continues to participate in the generation of latent variables for the entire frame at subsequent preset inference positions. At the subsequent inference positions, the local trajectory offset judgment and the current valid benchmark formation judgment are re-executed. This avoids prematurely fixing local trajectory units that are still in the process of normal structure formation as current valid benchmarks.

[0073] When a local trajectory unit lacks a complete predictive clean latent variable fragment, has incomplete boundary intersection data, or is missing a conditional response fragment, it is not considered a current valid benchmark. For local trajectory units lacking a conditional response fragment but with complete other data, only the local change and boundary intersection combination judgments in the local trajectory offset judgment are performed; it is not directly considered a current valid benchmark. For local trajectory units lacking boundary intersection data, boundary intersection relationship judgments are not performed, and the local trajectory unit is not registered as a current valid benchmark. The corresponding judgments are re-executed after complete data is obtained for subsequent inference positions.

[0074] When the same local trajectory unit again meets the current valid reference formation conditions at a subsequent inference position, the image generation device replaces the previously saved current valid reference of the local trajectory unit with the newly formed predicted clean latent variable fragment at the current inference position, while retaining the new inference position identifier, boundary intersection data, main condition response identifier, and corresponding condition response value. Subsequently, when performing local trajectory backreference in S4, the device prioritizes reading the current valid reference most recently formed from the current inference position, and does not always use the data formed the first time.

[0075] Through the above processing, the image generation device can form a unified and clear state distinction for all local trajectory units in the same low-latency image generation process. Local trajectory units that have met the combined anomaly conditions are registered as local trajectory offset units; local trajectory units that have not experienced local trajectory offset and meet the three types of boundary requirements form the current valid reference; the remaining local trajectory units that do not meet the conditions for forming the current valid reference are registered as areas to be observed.

[0076] Both the local trajectory offset unit and the current valid reference are derived from the actual generated data changes between adjacent inference positions, providing direct data and status basis for subsequent local trajectory back-reference based on the current valid reference.

[0077] S4 specifically refers to: After completing the registration of local trajectory offset units in S3, the image generation device first reads the position of the local trajectory offset unit corresponding to the current image generation task, and retrieves the corresponding current valid reference based on the local trajectory unit number. The current valid reference comes from the predicted clean latent variable fragments that were not registered as local trajectory offset units in the same image generation process, and whose local changes are within the corresponding local change boundary range, whose boundary intersection amounts formed by each connection direction are within the corresponding boundary intersection boundary range, whose main conditional response identifiers corresponding to the previous inference position and the current inference position are consistent, and whose conditional response change amounts are within the corresponding conditional response boundary range. These fragments do not come from pre-stored fixed image templates, nor do they call data generated by other image generation tasks.

[0078] When the same local trajectory offset unit forms multiple current valid references at multiple preceding inference positions, the image generation device selects the reference based on the order inference positions formed by the current inference position and the current valid references, prioritizing the reading of the most recently formed current valid reference from the current inference position. The distance between inference positions is determined according to the preset inference position order in the sampling configuration file, and is not based on changes in object position, texture, or manual judgment in the generated image. For example, when the current position is the third inference position, the current valid reference formed at the second inference position is read first; if no current valid reference is formed at the second inference position, then the current valid reference formed at the first inference position is read.

[0079] After reading the current valid reference, the image generation device further reads the noisy latent variable data of the entire image before entering the current inference position, as well as the noise control parameters corresponding to the current inference position. According to the fixed spatial coordinates of the local trajectory offset unit, the image generation device extracts the local noisy latent variable fragment corresponding to the position of that local trajectory offset unit from the noisy latent variable data, and uses this local noisy latent variable fragment as the data to be decomposed to form the current local noise residual data. Noisy latent variable data at the corresponding positions of other local trajectory units are not read.

[0080] The current local noisy latent variable segment, the subsequently formed current local noise residual data, and the current effective benchmark maintain the same fixed spatial position relationship of local trajectory units, so that the subsequently formed local back-reference latent variables always correspond to the fixed spatial range of the current local trajectory offset unit.

[0081] Subsequently, the image generation device reads the noise control parameters corresponding to the current inference position. These noise control parameters originate from the sampling configuration file corresponding to the latent consistency image generation model and correspond one-to-one with the current preset inference position. The image generation device then forms local backreference latent variables based on the current valid baseline, the current local noise residual data, and the noise control parameters at the current inference position.

[0082] Specifically, the current effective benchmark provides the source of structural information after local trajectory verification, the current local noise residual data provides the source of noise change at the current inference position for this image generation task, and the noise control parameters at the current inference position limit the noise position of the current local latent variable.

[0083] The latent variable data in the current effective benchmark are directly used as the clean structural components of the corresponding local trajectory units according to the fixed spatial coordinates and latent variable channels of the local trajectory units, without extracting structural data from other inference locations or other image generation tasks.

[0084] The image generation device, according to the noise control parameters corresponding to the current inference position, uses the noisy latent variables of the current whole image before entering the current inference position as the data to be decomposed. It uses the predicted clean latent variable segment at the fixed spatial position of the corresponding local trajectory offset unit from the predicted clean latent variables obtained by the latent consistency image generation model at the current inference position as the current clean structure reference. Based on the noise state decomposition relationship corresponding to the model sampling configuration, the noise portion of this noisy latent variable, excluding the component corresponding to the current clean structure reference, is extracted as the current local noise residual data. The current clean structure reference is only used to obtain the noise residual of the current generation task at the current inference position and is not used as the source of the restored structure for local backreference; the source of the restored structure for local backreference is still the last valid reference formed before the current inference position.

[0085] When forming local backreference latent variables, the structure preservation coefficient and noise preservation coefficient corresponding to the current inference position in the sampling configuration are applied to the clean structure component of the current effective reference and the current local noise residual data, respectively, and are combined according to the same spatial coordinates and latent variable channels, so that the combined result is in the same noise state as the current whole latent variable.

[0086] The local back-reference latent variables formed according to the above noise state decomposition and corresponding combination relationship maintain the same fixed spatial position, latent variable channel aperture and noise state corresponding to the current whole latent variable, and are used as the write data of the recovery area inside the current local trajectory offset unit.

[0087] After the local backreference latent variables are formed, they are not directly overwritten with the current full-scale latent variables. Instead, the internal recovery region and the boundary junction region are first divided according to the spatial position of the local trajectory unit determined by S1. The internal recovery region corresponds to the central region of the local trajectory offset unit and is used to write the local backreference latent variables; the boundary junction region corresponds to the position where the local trajectory offset unit connects with the adjacent region and is used to handle the data connection between the local backreference region and the non-offset region.

[0088] After the internal recovery area is determined, the regions directly adjacent to the local trajectory offset units that are not registered as local trajectory offset units are located according to their fixed spatial positions. The latent variable data corresponding to these adjacent local trajectory units at the current inference position are then obtained. The actual reading direction is based on the spatial connection relationship determined by S1. Directions with adjacent local trajectory units on the upper, lower, left, and right sides are all included in the processing; directions located at the edge of the entire latent variable and without adjacent units on the outside are not read.

[0089] When the boundary is closed, the edge data of the local latent variable near the boundary position and the edge data of the current latent variable of the adjacent unoffset local trajectory unit in the same connection direction are obtained respectively. After the data on both sides are aligned with the fixed position in the boundary junction area, the proportion of each participating in the writing of the boundary junction area is determined according to the already saved boundary continuous configuration, and a corresponding combined weight is formed accordingly to maintain a continuous data transition between the internal recovery area and the surrounding unoffset area.

[0090] The boundary continuity configuration used is determined before the deployment of the current potential consistency image generation model. During validation, normal generated samples containing continuous contours, object edges, text strokes, and slender structures are selected. The boundary changes of adjacent local trajectory units at different inference positions are recorded, and the data ratio relationship used for boundary intersection is determined from samples in a normal continuous state. The resulting configuration is saved in the local trajectory configuration file corresponding to the current model and can be directly read and used at runtime.

[0091] During operation, when the boundary data between the local reference latent variable and the current latent variable of the adjacent non-offset local trajectory unit is within the boundary continuity configuration range, the continuity relationship of the current boundary junction area is maintained; when the boundary data between the edge of the local reference latent variable and the edge of the adjacent non-offset area exceeds the corresponding range of the current boundary continuity configuration, the local reference range is not expanded, but the boundary junction area is transitioned according to the junction rules in the current boundary continuity configuration.

[0092] For local trajectory offset units located at the edge of the entire latent variable, since there are no adjacent local trajectory units on the outside, only the actual inner boundary intersection area is closed. For continuous local trajectory offset areas where there are no valid unoffset local trajectory units around, no boundary closure is performed, and the formed local backreference latent variable is retained and enters the next preset inference position.

[0093] After completing local backtracking and boundary closure, the image generation device writes the processed local latent variables back to the corresponding positions of the current whole-image latent variables according to the fixed spatial coordinates of the local trajectory units. The write-back range is limited to the positions that have been registered as local trajectory offset units, and the current latent variables of unregistered local trajectory offset units are not modified.

[0094] The processed whole latent variable retains the spatial size and number of channels specified by the current model, and its noise state corresponds to the current inference position. After the local data is written back, the whole latent variable is directly sent to the next preset inference position to continue generation. The local trajectory offset unit is not split and processed separately, nor does it increase the number of generation times.

[0095] When a local trajectory offset cell does not have a current valid reference, local backreference processing is not performed, and the original latent variables corresponding to the current local trajectory offset cell are retained to continue participating in the subsequent whole-frame generation process. When the spatial dimensions of the current valid reference cell and the local trajectory offset cell are inconsistent, noise residual data is missing, noise control parameters are missing, or boundary intersection data is missing, local backreference is also not performed to avoid using data that cannot correspond to the current inference position to modify latent variables.

[0096] Through the above processing, the local trajectory offset unit can utilize the current effective benchmark that has been formed and verified in the same image generation process to form local back-reference latent variables while maintaining consistency with the current noise source, current inference position, and spatial coordinates. The boundary handover process is completed through adjacent non-offset local trajectory units, so that the local trajectory recovery process is still in the original low-latency whole-frame latent variable generation process.

[0097] S5 specifically refers to: After completing the local backreference and boundary closure processing in S4, the image generation device uses the whole latent variable after local latent variable replacement as the continued input data for the current image generation task, and continues to perform latent consistency image generation processing according to the next preset inference position registered in the sampling configuration file.

[0098] When entering the next preset inference position, the image generation device does not reread the image generation conditions, regenerate the initial noise, or re-establish the local trajectory unit partitioning relationship. The current input data is the whole latent variable after S4 processing. This whole latent variable maintains the spatial size, number of latent variable channels, and noise state corresponding to the current latent consistency image generation model.

[0099] The noise control parameters used for the next preset inference position are obtained according to the records in the current model sampling configuration file and correspond to the predetermined inference position order. The number of local trajectory offset units, their locations, and the number of times local backreferences have been performed do not change the number of subsequent preset inference positions, nor do they adjust the corresponding noise control parameters. The entire local recovery process still follows the original few-step generation order of the current model.

[0100] Upon reaching the next preset inference position, the locally processed whole latent variables are input into the latent consistency image generation model, and a whole generation process is completed according to this inference position. The regions that have already undergone local backreference are still used as the corresponding parts in the whole latent variables in this generation, and are not divided into new generation tasks, nor are additional generation branches set for local trajectory offset units.

[0101] After completing the current preset inference position generation process, the image generation device reads the predicted clean latent variable fragment, boundary junction data and condition response data corresponding to the current inference position again according to the fixed spatial coordinates of the local trajectory unit determined in S2, and re-determines the state of each local trajectory unit according to the local trajectory offset judgment rule in S3.

[0102] For a local trajectory offset unit that has already undergone local backreference processing at the previous inference position, when the same local trajectory unit at the current inference position meets the requirements of local change boundary, boundary intersection boundary, and conditional response boundary, the image generation device removes the local trajectory offset unit identifier corresponding to the local trajectory unit and re-forms the current valid reference based on the current inference position.

[0103] The conditions for removing the local trajectory offset unit identifier include: the change in the predicted clean latent variable within the current local trajectory unit is within the local change boundary range corresponding to the current inference position; the boundary intersection amounts formed by all directions of adjacent local trajectory units are within the boundary intersection boundary range corresponding to the current inference position; and the main condition response identifiers corresponding to the previous inference position and the current inference position are consistent, and the corresponding condition response change amounts are within the condition response boundary range corresponding to the current inference position. When all the above conditions are met simultaneously, the corresponding local trajectory offset unit identifier is removed; if not, the status judgment continues according to the subsequent preset inference position.

[0104] For regions that still meet the conditions for forming local trajectory offset after performing local backreference at the previous inference position, the image generation device does not continuously use the same current valid reference to perform repeated backreference. Instead, it first reads the local change state, boundary intersection state, and condition response state that are re-formed at the current inference position, and determines whether the current local trajectory unit still has a current valid reference.

[0105] When the current local trajectory unit still has a local trajectory offset unit state, and there is a current valid reference formed last according to the preset inference position order before the current inference position, the local backreference latent variable corresponding to the current inference position is formed again according to the local backreference process in S4; when there is no current valid reference, the current whole latent variable is retained and continues to enter the next preset inference position, without calling unverified data to replace the current generated result.

[0106] When multiple adjacent local trajectory units are simultaneously registered as local trajectory offset units, the image generation device determines the adjacency relationship based on the fixed spatial coordinate relationship in S1, and processes local trajectory offset units with continuous spatial connection as continuous local trajectory offset regions. Boundary closure judgment is not repeatedly performed between adjacent local trajectory offset units within a continuous local trajectory offset region; instead, closure processing is performed based on the boundary intersection relationship between the region's outer perimeter and the unoffset local trajectory units.

[0107] For locations where there are unoffset local trajectory units outside a continuous local trajectory offset region, the image generation device reads the current latent variable corresponding to the boundary intersection region and performs boundary closure processing based on the boundary continuously configured in S4; for directions where there are no adjacent unoffset local trajectory units, no additional boundary region is constructed and no virtual region closure is performed.

[0108] The number of preset inference positions is determined according to the sampling configuration of the current potential consistency image generation model. Taking this embodiment as an example, which uses four preset inference positions, the generation process proceeds sequentially through the first, second, third, and fourth preset inference positions according to the order registered in the sampling configuration.

[0109] The preceding inference positions are primarily used to obtain predicted clean latent variables and form a current effective baseline for subsequent use. Subsequent inference positions continue to generate the entire latent variable dataset and perform corresponding recovery processing when local trajectory shifts are detected. The actual number of inference positions used depends on the current model and its sampling configuration; the four preset inference positions are only used as a specific configuration in this embodiment.

[0110] After completing the generation of all latent variables at the last preset inference position, the image generation device obtains the final cleaned latent variables. The final cleaned latent variables are derived from the model output after the last complete generation process and serve as the data source for subsequent latent variable decoding.

[0111] When there are still local trajectory offset units at the last preset inference position, the image generation device reads the current effective reference formed by the local trajectory unit in the order of preset inference positions before the last preset inference position. The image generation device obtains the boundary intersection amount of the corresponding connection direction based on the boundary intersection data corresponding to the current effective reference and the final clean latent variable edge data corresponding to the adjacent non-offset local trajectory units in the last preset inference position, according to the boundary intersection amount determined in S3. At the same time, it compares the main condition response identifier corresponding to the current effective reference with the main condition response identifier corresponding to the last preset inference position. When the two main condition response identifiers are consistent, the condition response change amount is formed based on the condition response values ​​corresponding to the two.

[0112] When all actual connection directions correspond to boundary intersection quantities within the boundary intersection boundary range of the last preset inference position, the two main condition response identifiers are consistent, and the resulting condition response change quantity is within the condition response boundary range of the last preset inference position, the current valid reference is confirmed to have the final recovery access condition; if the above conditions are not met simultaneously, the current valid reference is not confirmed to have the final recovery access condition.

[0113] The current valid benchmark is used for reassessment before final recovery and is only used as historical comparison data in the final recovery admission assessment. The boundary handover data, main condition response identifiers and condition response values ​​corresponding to the current valid benchmark are not registered as data regenerated at the last preset inference position. The edge data, main condition response identifiers and condition response values ​​of the last preset inference position used in the final recovery admission assessment are all derived from the actual generation results of the last preset inference position.

[0114] When a valid local trajectory offset unit corresponding to the last preset inference position exists, and the valid local trajectory offset unit has been confirmed as meeting the final recovery admission criteria, the image generation device performs final recovery processing only on the internal region of the registered local trajectory offset unit, without changing other normally generated regions. During the final recovery process, the predicted clean latent variable fragment corresponding to the current valid local trajectory offset unit is written into the internal region of the local trajectory offset unit, and the edge data of the recovered internal region and the edge data of the final clean latent variable corresponding to the adjacent non-offset local trajectory units are read. Boundary closure is performed based on the spatial correspondence between the two in the boundary junction region determined in S1, so that the recovered region and the final clean latent variable maintain a spatial continuity.

[0115] If the local trajectory offset unit corresponding to the last preset inference position does not have a currently valid reference, or if the existing currently valid reference has not passed the final recovery admission judgment, then the final recovery will not be performed on the region, and the final clean latent variable formed by the last complete generation process will still be used as the subsequent decoding object.

[0116] After the final clean latent variables are determined, the corresponding latent variable decoding parameters are read according to the current latent consistency image generation model configuration, and pixel-level image conversion is completed accordingly. This decoding is only performed on the final clean latent variables; the predicted clean latent variables obtained at each preset inference position in the intermediate stages remain in latent variable form and are not decoded at the pixel level separately.

[0117] In practice, the image generation device receives image generation conditions that include people, thin rod structures, and architectural text areas. The target output size is set to 1024 pixels by 1024 pixels, and the current potential consistency image generation model is configured with four preset inference positions.

[0118] The first preset inference position completes the initial prediction clean latent variable formation; the second preset inference position obtains the local data corresponding to the first preset inference position, and completes the adjacent inference position judgment according to S3, and registers the current valid reference for the local trajectory unit that meets the current valid reference formation conditions; when the third preset inference position detects that there is a local change anomaly and boundary mismatch state in the local trajectory unit corresponding to the thin rod structure, it reads the current valid reference formed by the second preset inference position, and forms a local back-reference latent variable according to the noise control parameters corresponding to the third preset inference position; the fourth preset inference position continues to complete the generation of the entire latent variable and the final decoding.

[0119] Through the above process, the local trajectory offset processing is always within the original inference process of the potential consistency image generation model. It does not change the number of preset inference positions, does not increase the number of independent generation of local abnormal regions, and does not rely on external reference images. Instead, it uses the already formed and verified current effective benchmark to restore the local trajectory offset region in the same image generation process, thereby improving the ability to maintain local structure in the low-latency image generation process.

[0120] The above content is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A low-latency image generation method based on latent consistency constraints, characterized in that, Follow these steps: S1. Read the image generation conditions, target image size, latent variable space parameters corresponding to the latent consistency image generation model, and preset inference position to form initial latent variables, and divide the local trajectory units and the boundary junction areas between adjacent local trajectory units according to the fixed positions in the latent variable space. S2. Generate the entire latent variable according to the preset inference position, obtain the predicted clean latent variable and conditional response data at each inference position, and extract the corresponding predicted clean latent variable fragment, boundary junction data and conditional response fragment according to the fixed position of the local trajectory unit. S3. Based on the changes in the predicted clean latent variables of the same local trajectory unit in adjacent inference positions, the boundary intersection relationship of the corresponding directions of adjacent local trajectory units, and the main condition response identifier and condition response change amount of the corresponding image generation condition constraints, register the local trajectory offset unit, and register the predicted clean latent variable segments that have not been registered as local trajectory offsets but satisfy the local change boundary, boundary intersection boundary and condition response boundary as the current valid benchmark. S4. For local trajectory offset units, read the last formed current effective reference before the current inference position, the local noise residual data corresponding to the current inference position, and the noise control parameters. Based on the noise control parameters, map the latent variable structure components of the current effective reference to the current noise state, and combine them with the local noise residual data to form local back-reference latent variables at the same noise position. Then, based on the current latent variables of adjacent non-offset local trajectory units, perform closure on the boundary intersection area determined by S1. S5. After completing the local backreference and boundary closure, the entire latent variable is sent to the next preset inference position. The remaining preset inference positions are processed according to S2 to S4. After obtaining the final clean latent variable at the last preset inference position, the latent variable decoding is performed.

2. The low-latency image generation method based on potential consistency constraints according to claim 1, characterized in that, S1 includes: The latent variable spatial size is determined based on the target image size and the spatial scaling ratio corresponding to the latent consistency image generation model. Initial latent variables with the same latent variable spatial size and number of latent variable channels are formed based on the random noise seed corresponding to the current image generation task. Local trajectory units are divided according to fixed coordinates in the latent variable space, and corresponding boundary junction regions are established between local trajectory units that have vertical and horizontal adjacent relationships. The fixed coordinates of the local trajectory unit and the position of the boundary intersection region remain unchanged across all preset inference positions in the current image generation task.

3. The low-latency image generation method based on potential consistency constraints according to claim 1, characterized in that, S2 include: After completing the generation of the entire latent variable at each preset inference position, the current predicted clean latent variable is saved in correspondence with the current inference position identifier; Based on the fixed coordinates of the local trajectory unit, extract the corresponding predicted clean latent variable fragments and boundary junction data from the current predicted clean latent variables, and extract the corresponding condition response fragments from the condition response data at the current inference position. Establish a correspondence between local trajectory unit numbers, current inference position identifiers, predicted clean latent variable segments, boundary handover data, and conditional response segments.

4. The low-latency image generation method based on potential consistency constraints according to claim 1, characterized in that, S3 includes: Read the predicted clean latent variable fragments corresponding to the same fixed spatial coordinates in the previous and current inference positions according to the local trajectory unit number; The changes between two predicted clean latent variable segments are determined based on the local change boundary corresponding to the current inference position; When the predicted changes in the cleaning latent variable exceed the local change boundary corresponding to the current inference position, a local change anomaly flag is registered for the corresponding local trajectory unit.

5. The low-latency image generation method based on potential consistency constraints according to claim 4, characterized in that, S3 also includes: For local trajectory units with direct spatial connections, boundary intersection amounts are formed according to the corresponding connection directions. When the boundary intersection amount formed in the corresponding connection direction exceeds the boundary intersection boundary corresponding to the current inference position, a boundary mismatch flag is registered in the corresponding direction. Compare the primary condition response identifiers corresponding to the same local trajectory unit in the previous inference position and the current inference position. If the primary condition response identifiers corresponding to the two inference positions are inconsistent, register the condition migration flag. If the primary condition response identifiers corresponding to the two inference positions are consistent, form the corresponding condition response change amount. Register the condition migration flag when the condition response change amount exceeds the condition response boundary corresponding to the current inference position. When the local change anomaly flag and the boundary mismatch flag are both set, the local trajectory offset unit is registered; when the local change anomaly flag and the condition migration flag are both set, the local trajectory offset unit is registered. The predicted clean latent variable fragments that are not registered local trajectory offset units, whose local changes are within the corresponding local change boundary, whose boundary intersection amounts formed by each corresponding connection direction are within the corresponding boundary intersection boundary, whose main conditional response identifiers corresponding to the previous inference position and the current inference position are consistent, and whose conditional response changes are within the corresponding conditional response boundary, are registered as the current valid benchmarks.

6. The low-latency image generation method based on potential consistency constraints according to claim 1, characterized in that, S4 include: Retrieve the currently valid reference formed before the current inference position according to the local trajectory unit number of the local trajectory offset unit, and select the last currently valid reference formed before the current inference position according to the preset inference position order; Read the local noise residual data corresponding to the current inference position according to the fixed spatial coordinates of the local trajectory offset unit, and read the noise control parameters corresponding to the current inference position; Using the current valid benchmark as the source of the clean structure and the local noise residual data as the noise source of the current generation task, a local back-reference latent variable is formed that is at the same noise position as the current whole latent variable.

7. The low-latency image generation method based on potential consistency constraints according to claim 6, characterized in that, S4 also includes: The local trajectory offset unit is divided into an internal recovery region and a boundary junction region located outside the internal recovery region. The local back-reference latent variable is written into the internal recovery region. Read the current latent variables of adjacent unoffset local trajectory units that have a direct spatial connection with the local trajectory offset unit; Based on the continuous configuration of the boundary, determine the combined weights corresponding to the edge data of the local latent variable and the edge data of the current latent variable of the adjacent non-offset local trajectory unit. Perform closure on the boundary junction area according to the combined weights, and write the local latent variable after closure back to the fixed spatial position corresponding to the current whole latent variable.

8. The low-latency image generation method based on potential consistency constraints according to claim 1, characterized in that, S5 include: The entire latent variable after local backreference and boundary closure is used as the input for the next preset inference position, while keeping the current image generation conditions, local trajectory unit fixed coordinates and preset inference position order unchanged; At the next preset inference position, the predicted clean latent variables, boundary junction data and condition response data are obtained again, and the local trajectory offset judgment is re-executed; When the local trajectory unit after local backtracking has a local change within the corresponding local change boundary at the next preset inference position, the boundary intersection amount formed by each corresponding connection direction is within the corresponding boundary intersection boundary, the main condition response identifiers corresponding to the previous inference position and the next preset inference position are consistent, and the condition response change amount is within the corresponding condition response boundary, the corresponding local trajectory offset unit identifier is removed, and the predicted clean latent variable segment corresponding to the next preset inference position is updated to the current valid benchmark.

9. The low-latency image generation method based on potential consistency constraints according to claim 1, characterized in that, S5 also includes: After obtaining the final clean latent variable at the last preset inference position, the current valid reference formed last in the preset inference position order is read for the fixed spatial position that is still registered as a local trajectory offset unit before the last preset inference position. When the boundary intersection amount formed by the boundary intersection data corresponding to the current effective benchmark and the final cleaning latent variable data of the adjacent unoffset local trajectory unit corresponding to the last preset inference position is within the corresponding boundary intersection boundary range, and the main condition response identifier corresponding to the current effective benchmark is consistent with the main condition response identifier corresponding to the last preset inference position, and the condition response change amount formed by the corresponding condition response values ​​is within the corresponding condition response boundary range, the internal region of the corresponding local trajectory offset unit in the final cleaning latent variable is restored with the current effective benchmark, and the boundary intersection region is closed according to the final cleaning latent variable data corresponding to the adjacent unoffset local trajectory unit. Perform latent variable decoding on the final cleaned latent variables after local processing.

Images