An image generation method and device based on deep learning

Abstract

This invention belongs to the field of image processing technology and discloses an image generation device based on deep learning. The device comprises a conditional parsing and path scoring unit, a dynamic path selection and generation unit, a local region recognition and thinning unit, a multi-scale fusion discrimination and feedback unit, and an image output and display unit. Through the joint design of the conditional parsing and path scoring unit and the dynamic path selection and generation unit, this invention achieves dynamic switching and adaptive control of the image generation path. The descriptive information input by the user not only affects the path score but also participates in the real-time structural configuration of the generation network, thereby automatically matching a suitable generation path according to different input conditions. This flexible path adjustment mechanism overcomes the limitations of fixed paths and a single generation process in existing image generation methods, improving the adaptability of the generation system to complex input requirements and effectively supporting personalized image generation.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically a method and apparatus for image generation based on deep learning. Background Technology

[0002] Existing image generation technologies, especially in the context of the rapid development of deep learning, widely adopt mainstream frameworks such as Generative Adversarial Networks (GANs) and diffusion models to achieve realistic image generation effects through large-scale data training.

[0003] However, these methods have many limitations in practical applications. First, current image generation methods rely heavily on model parameters obtained during offline training. The generation path is usually fixed in the model design stage, making it difficult to flexibly adjust the generation strategy according to different user input conditions. This results in insufficient adaptability of existing systems, making it difficult to cope with complex and dynamic image generation needs. Second, traditional methods mostly focus on optimizing the overall image quality, lacking targeted control over key local areas of the image. They cannot dynamically adjust key detail areas in real time during the generation process, which can easily lead to problems such as local blurring and loss of detail.

[0004] In addition, existing generative structures typically rely on a single-scale quality discriminator and lack a mechanism for synchronous monitoring and feedback of image quality at multiple scales. This can lead to local and global quality imbalances in the generated results in complex scenes, affecting the overall visual effect of the image. Summary of the Invention

[0005] The purpose of this invention is to provide an image generation method and apparatus based on deep learning to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: an image generation device based on deep learning, which comprises a condition parsing and path scoring unit, a dynamic path selection and generation unit, a local region identification and thinning unit, a multi-scale fusion discrimination and feedback unit, and an image output and display unit;

[0007] Condition parsing and path scoring unit: It is used to receive natural language descriptions, reference image samples and parameterized control vectors input by users, parse and generate condition vectors, and score the preset image generation path library in real time based on the condition vectors. The scoring results are also used as parameter inputs for subsequent path generation.

[0008] Dynamic Path Selection and Generation Unit: Connected to the Conditional Parsing and Path Scoring Unit, it is used to select the optimal image generation path based on the scoring results, dynamically configure the generation network structure based on the scoring results, and generate multi-scale image features layer by layer along the selected path. The scoring results drive the dynamic adjustment of path generation nodes, convolution kernel parameters, and skip structures in real time.

[0009] Local region recognition and thinning unit: connected to dynamic path selection and generation unit, used to detect key detail regions in real time during image generation, and dynamically adjust the local thinning intensity based on path scoring results. Pyramid convolutional network is used to perform high-resolution thinning of key regions, and the thinning results are fused with the original image features in real time.

[0010] Multi-scale fusion discrimination and feedback unit: connected to the local region recognition and refinement unit, it is used to perform parallel quality discrimination on the image generation results at the full-scale, local-scale and feature map scales, and to weight and fuse the discrimination results of each scale to form an overall quality feedback. The feedback is transmitted to the condition parsing and path scoring unit in real time for dynamic updating of path scoring parameters.

[0011] Image output and display unit: connected to the multi-scale fusion discrimination and feedback unit, used to output the final generated target image after the quality discrimination result reaches a preset threshold. The image output process must verify whether the quality discrimination score is qualified. If it is not qualified, it automatically returns to the condition parsing and path scoring unit to adjust the path scoring parameters and re-drive image generation.

[0012] Preferably, the condition parsing and path scoring unit includes:

[0013] (1) User input parsing and condition vector construction: This device is used to receive natural language descriptions, reference image samples and parameterized control vectors input by users. The system first performs semantic parsing, image feature extraction and parameter association mapping on the input content. Through cross-modal association technology, different types of inputs are uniformly parsed into structured condition vectors. The generated condition vectors comprehensively describe the user's intent, reference sample features and generation parameter constraints, and provide basic data support for subsequent path scoring and generation, ensuring full utilization and dynamic adaptation of input information.

[0014] (2) Real-time path scoring and dynamic generation control: Based on the generated condition vector, the system calls candidate paths from the preset image generation path library and scores them in real time through path matching degree, multi-dimensional feature fit degree and generation target deviation. The scoring results are not only used for current path optimization, but also dynamically fed back to adjust the subsequent path generation parameters, realizing a real-time closed loop of path scoring and generation control. The scoring process refers to the weighted comprehensive function, and the weight factor is dynamically adjusted according to the condition vector to ensure that the path selection meets the user's needs and the generation effect is stable and reliable.

[0015] The formula for expressing path scoring is:

[0016] ,

[0017] In the formula, path The scoring results (dimensionless) : Input the matching degree weight coefficient (range 0~1), Input conditions and path The matching degree (0~1, calculated using cosine similarity) Historical generation quality weighting coefficient (range 0~1, :path Historical image quality score (structural similarity index, SSIM, range 0~1). Generate complexity weight coefficients (range 0~1). :path The generation complexity (unit: FLOPs, floating-point computation).

[0018] Formula source: The scoring function is adapted from the weighted scoring model, referencing literature on dynamic scoring of image generation paths, and supported by publicly available domestic data.

[0019] Preferably, the dynamic path selection and generation unit includes:

[0020] (1) Optimal path selection and dynamic configuration of the generation network: This unit is connected to the condition parsing and path scoring unit. Based on the scoring results, it selects the optimal image generation path from the preset path and configures the structure layout of the generation network in real time based on the path. The layer depth, number of channels and feature transfer method of the generation network are dynamically adapted and adjusted according to the scoring results, so that the generation process is targeted. The scoring results are used as dynamic configuration parameters to ensure that the selected path is highly matched with the user input conditions, thereby improving the accuracy and diversity of image generation.

[0021] (2) Path-driven feature generation and convolutional structure adjustment: Guided by the selected image generation path, the system generates multi-scale image features layer by layer along the path, driving the dynamic adjustment of path generation nodes, convolutional kernel parameters and skip connection structures in real time; the adjustment of convolutional kernel parameters is controlled by a parameter mapping function based on scoring weights to ensure that the convolutional kernel size, stride and channel response characteristics are highly consistent with the current path score and dynamically respond to changes in input conditions; this path control mechanism effectively improves image detail preservation and structural coherence.

[0022] The formula for dynamically adjusting the convolution kernel parameters is:

[0023] ,

[0024] In the formula, The kernel size (in pixels) used in the current generated path. : Basic convolutional kernel size (unit: pixels) Convolution kernel adjustment factor (unit: pixels) The selected path score is consistent with the calculation result of Unit 1.

[0025] Formula source: The dynamic adjustment formula for convolutional kernels is based on the principle of adaptive parameter configuration of convolutional neural networks and references the dynamic parameter adjustment design of ResNet.

[0026] Preferably, the local region identification and refinement unit includes:

[0027] (1) Detection of key detail regions and local thinning control: This unit is connected to the dynamic path selection and generation unit to monitor key detail regions in the image generation process in real time; the system dynamically identifies key regions that affect image quality based on the current path score results and adjusts the local thinning intensity in real time; the local thinning intensity is dynamically calculated through the correlation adjustment function according to the score feedback, and the change of thinning intensity is linked with the path score weight to ensure that key regions obtain higher resolution detail processing and improve the overall image quality;

[0028] The formula for adjusting the local refinement intensity is:

[0029] ,

[0030] In the formula, Current local refinement intensity parameters (unit: pixels), Base refinement intensity (unit: pixels), Refine intensity adjustment factor (unit: pixels), The selected path's score result (consistent with the calculation result in Unit 1);

[0031] Formula source: The thinning intensity dynamic adjustment method is based on the parameter control of the local saliency detection algorithm, and references the thinning design of the pyramid convolutional network.

[0032] (2) Pyramid refinement and feature fusion mechanism: For key areas, the system calls the pyramid convolutional network to refine layer by layer and generate multi-scale high-resolution features; during the refinement process, the local convolutional kernel parameters and stride are dynamically adjusted under the drive of refinement intensity, and the refinement result is fused with the original image features in real time to ensure that the details are highly consistent with the overall structure; the refinement intensity adjustment relies on the hierarchical mapping strategy of the scoring results to dynamically match the refinement degree with the path selection, thereby improving the local detail restoration and generation continuity of the image.

[0033] Preferably, the multi-scale fusion discrimination and feedback unit includes:

[0034] (1) Parallel quality discrimination and scoring feedback at all scales: This unit is connected to the local region recognition and thinning unit to perform parallel quality discrimination at the full-scale, local key scale and feature map scale in the image generation process; the system evaluates the sharpness, structural consistency and detail restoration at each scale to form multi-dimensional quality indicators; the quality discrimination results at each scale are weighted and fused based on the weight adjustment function, and the fusion results are used to construct global quality feedback to ensure that the quality evaluation is comprehensive and accurate and has real-time response capability;

[0035] (2) Quality feedback drives dynamic update of path scoring parameters: The overall quality feedback result after fusion will be transmitted to the condition parsing and path scoring unit in real time. As an important input for dynamic adjustment of path scoring parameters, the system optimizes the path scoring parameters step by step through the quality feedback mapping function, so that the path selection and generation strategy continuously fits the changes in image quality. The quality scoring fusion process refers to the weighted integral mechanism. The fusion weight is adjusted in real time with the image generation process to ensure that the feedback process is timely and targeted, and improves the adaptive capability of the generated path.

[0036] The formula for expressing the quality score fusion is:

[0037] ,

[0038] In the formula, Overall image quality score after fusion (unit: dimensionless, range 0~1) :No. The rating weights of the scale (weight coefficients, summing to 1), :No. Quality score of the scale (unit: dimensionless, range 0~1), Total number of scales to be judged (e.g., 3 scales).

[0039] Formula source: The multi-scale weighted scoring formula is derived from the image quality assessment weighted fusion principle, with reference to the SSIM multi-scale structure design.

[0040] Preferably, the image output and display unit includes:

[0041] (1) Image output judgment and quality verification process: This unit is connected to the multi-scale fusion discrimination and feedback unit. After the image is generated, the fusion quality score is judged based on the preset quality threshold. The system uses a quality verification function to check the current image item by item to ensure that the output image meets the quality requirements in terms of clarity, structural consistency and detail integrity. The image can only enter the final output process when the discrimination score meets the output conditions to ensure that the image result meets the expected standard.

[0042] The formula for determining image output is:

[0043] ,

[0044] In the formula, Output: whether the image is allowed to be output (Pass: allowed; Reject: not allowed). Integration quality score (from Unit 4) Preset quality threshold (range 0~1);

[0045] Formula source: The output condition determination formula originates from the closed-loop quality control process and conforms to the dynamic image quality approval mechanism;

[0046] (2) Adaptive path adjustment when quality is not up to standard: When the image quality judgment result is lower than the set threshold, the system automatically triggers the path adjustment process and transmits the current quality feedback back to the condition analysis and path scoring unit in real time; the path scoring parameters are dynamically updated according to the quality deviation, the optimal image generation path is reselected, and a new round of image generation process is driven; the path adjustment is iterated in multiple rounds according to the scoring correction mapping relationship to ensure continuous optimization of the generation path, and finally outputs a high-quality image, forming a closed-loop control.

[0047] This invention also provides a deep learning-based image generation method, which is based on the above-mentioned unit, and the specific steps of the method are as follows:

[0048] S1, Condition Parsing and Path Scoring: Receives natural language description, reference image and parameter vector input by the user, parses them into structured condition vector, and scores the preset image generation path library in real time based on the path scoring algorithm. The scoring result is used as the parameter input for subsequent path generation.

[0049] S2, Path Selection and Local Refinement Generation: Select the optimal generation path based on the results of condition parsing and path scoring, dynamically configure the generation network structure based on the scoring results, generate multi-scale image features layer by layer along the selected path, detect key detail regions in real time, and adjust the local refinement intensity synchronously based on the path score to perform high-resolution refinement on key regions.

[0050] S3, Conditional Analysis and Path Scoring: The quality judgment of the path selection and local refinement generated image is performed in parallel at the full-scale, local-scale and feature map scales. The judgment results of each scale are fused to form an overall quality feedback. The feedback results are transmitted to step one in real time to adjust the path scoring parameters. If the judgment score is lower than the quality threshold, it automatically returns to step one to re-execute. If the judgment score meets the requirements, the final generated image is output.

[0051] Preferably, the specific steps of condition parsing and path scoring in S1 are as follows:

[0052] S11, User Input Parsing and Condition Vector Construction: The system receives natural language descriptions, reference image samples, and parameter control vectors from user input. Semantic features are extracted from the natural language, image content is analyzed from the reference images, and numerical normalization is performed on the parameter vectors. Through cross-modal parsing and association mapping, the system transforms multi-source inputs into unified structured condition vectors. The condition vectors fully represent user intent, reference sample features, and generation control requirements, providing accurate input basis for subsequent path scoring and generation processes.

[0053] S12, Path Library Scoring and Dynamic Configuration of Parameter Input: The system uses the structured condition vector obtained from parsing to score candidate paths in the preset image generation path library in real time. The path scoring process comprehensively considers the path feature matching degree, parameter adaptability, and historical generation effect, and dynamically calculates the optimal score of each path. The scoring results are used as parameter inputs for path generation in real time, driving the dynamic selection and structural configuration of subsequent image generation paths, ensuring that the selected path is highly matched with the user input, and achieving efficient and accurate image generation control.

[0054] Preferably, the specific steps for path selection and local refinement generation in S2 are as follows:

[0055] S21, Optimal Path Selection and Dynamic Configuration of Generation Network: Based on the condition parsing and path scoring results, the system selects the optimal image generation path and dynamically configures the hierarchical structure, channel parameters and connection methods of the generation network based on the scoring results; multi-scale image features are generated layer by layer along the selected path to ensure that the generation process is structurally reasonable and that the generation path is highly consistent with the user input, thereby improving the overall image quality and generation efficiency.

[0056] S22, Key Detail Detection and Local High-Resolution Refinement: During the layer-by-layer generation of the image, the system detects key detail areas in real time, focuses on identifying important image content, and adjusts the local refinement intensity synchronously based on the path scoring results. It also adopts a multi-scale high-resolution refinement strategy to refine key areas, ensuring rich details and structural integrity, thereby improving the local image quality and visual realism.

[0057] Preferably, the specific steps of condition parsing and path scoring in S3 are as follows:

[0058] S31, Multi-scale quality discrimination and fusion feedback: The system performs quality discrimination on the image generated by path selection and local refinement in parallel at the full-image scale, the local key region scale, and the feature map scale; the quality indicators of each scale are fused in real time to form a comprehensive overall quality feedback; the feedback result is synchronously transmitted to the path scoring parameter adjustment unit to dynamically optimize the path selection and generation parameters, ensuring accurate and timely quality evaluation.

[0059] S32, Quality Threshold Verification and Generation Process Control: The system performs threshold judgment on the quality feedback results after fusion. If the quality score is lower than the preset standard, it immediately returns to the path scoring step, reselects the generation path, and executes the image generation process. If the judgment score meets the quality requirements, it outputs the final generated target image, ensuring that the generated result meets the user's expectations and forming an efficient closed-loop quality control process.

[0060] The beneficial effects of this invention are as follows:

[0061] 1. This invention achieves dynamic switching and adaptive control of image generation paths through the joint design of condition parsing and path scoring units, as well as dynamic path selection and generation units. The descriptive information input by the user not only affects the path score but also participates in the structural configuration of the generation network in real time, thereby automatically matching a suitable generation path according to different input conditions. This flexible path adjustment mechanism breaks through the limitations of fixed paths and simple generation processes in existing image generation methods, improves the adaptability of the generation system to complex input requirements, and effectively supports personalized image generation.

[0062] 2. By setting up a local region recognition and thinning unit, this invention can actively detect and identify key detail regions in the image in real time during the image generation process. For the detected key regions, a pyramid convolutional network is used for high-resolution thinning. By dynamically linking the thinning intensity with the path scoring results, it ensures that the detail processing is more in line with the overall image style and user input conditions. This design effectively avoids the problems of local image distortion and detail blurring in the prior art, and significantly improves the fidelity and realism of the generated image at the detail level.

[0063] 3. This invention employs a multi-scale fusion discrimination and feedback unit to support simultaneous discrimination and real-time feedback of image generation quality at the full-scale, local-scale, and feature map scales. The discrimination results at each scale are weighted and fused, serving as an important basis for adjusting the generation path and continuously optimizing the image generation process. By establishing a closed-loop control mechanism for path scoring and discrimination feedback, a dynamic balance between global quality and local details is achieved, effectively ensuring the stability and consistency of the overall quality of the generated image and avoiding the problem of uncontrollable generation quality. Attached Figure Description

[0064] Figure 1 This is a flowchart of the deep learning-based image generation device of the present invention;

[0065] Figure 2 This is a flowchart of the deep learning-based image generation method of the present invention. Detailed Implementation

[0066] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0067] like Figures 1 to 2 As shown, this embodiment of the invention provides an image generation device based on deep learning. The device consists of a condition parsing and path scoring unit, a dynamic path selection and generation unit, a local region recognition and thinning unit, a multi-scale fusion discrimination and feedback unit, and an image output and display unit.

[0068] Condition parsing and path scoring unit: It is used to receive natural language descriptions, reference image samples and parameterized control vectors input by users, parse and generate condition vectors, and score the preset image generation path library in real time based on the condition vectors. The scoring results are also used as parameter inputs for subsequent path generation.

[0069] Implementation Example of Conditional Parsing and Path Scoring Unit: In this embodiment, the conditional parsing and path scoring unit includes: a natural language parsing module, an image sample feature extraction module, and a path scoring calculation module; the natural language parsing module uses a BERT pre-trained language model to perform vector encoding on the user-input text description, with an encoding dimension of 512 dimensions; the image sample feature extraction module uses a convolutional neural network based on ResNet-50 to extract feature maps from the input reference image, with an output image feature dimension of 2048 dimensions;

[0070] The two inputs mentioned above are concatenated and then fed into the path scoring calculation module. The scoring calculation is performed according to the following steps: First, the cosine similarity between the input conditions and each preset path feature is calculated, with a matching degree range of 0 to 1; second, the historical generation quality records of the path are retrieved (using the structural similarity index SSIM, with a preset range of 0.7 to 0.98); then, the path computational complexity parameter is extracted (in units of FLOPs, with a preset path FLOPs range of 30G to 80G); finally, the path scoring result is calculated according to the weight ratio; the weight ratio is set to 0.5:0.3:0.2 during actual training; the scoring result is not only used for path ranking but also serves as the configuration parameter for the convolution kernel of subsequent generation units.

[0071] Dynamic Path Selection and Generation Unit: Connected to the Conditional Parsing and Path Scoring Unit, it is used to select the optimal image generation path based on the scoring results, dynamically configure the generation network structure based on the scoring results, and generate multi-scale image features layer by layer along the selected path. The scoring results drive the dynamic adjustment of path generation nodes, convolution kernel parameters, and skip structures in real time.

[0072] Implementation Example of Dynamic Path Selection and Generation Unit: In this embodiment, the dynamic path selection and generation unit consists of a path selection module and a generation network module. The path selection module automatically calls the highest-scoring generation path based on the scoring results. This path is based on an improved UNet network structure and supports dynamic skip connection configuration. The convolution kernel parameters are transmitted in real time from the scoring results of the condition parsing unit. When the score is high, a larger convolution kernel is automatically enabled (the basic kernel size is 3×3, the adjustment stride is the score value × 2, and the convolution kernel size can be adaptively switched between 3×3 and 9×9). The generation network adopts a four-layer convolutional structure. The basic feature extraction stage supports dynamic activation of nodes between paths, and the number of network layers can be automatically increased by one or two convolutional layers according to the score. The path scoring results are synchronously transmitted to the local region recognition unit for subsequent refinement intensity adjustment. The generation process is executed in real time under the NVIDIA RTX 3090 GPU environment, and the average generation time per image is approximately 0.7 seconds.

[0073] Local region recognition and thinning unit: connected to dynamic path selection and generation unit, used to detect key detail regions in real time during image generation, and dynamically adjust the local thinning intensity based on path scoring results. Pyramid convolutional network is used to perform high-resolution thinning of key regions, and the thinning results are fused with the original image features in real time.

[0074] Implementation Example of Local Region Recognition and Refinement Unit: In this embodiment, the local region recognition and refinement unit includes a saliency detection module, a refinement intensity adjustment module, and a pyramid convolutional refinement network. The saliency detection module adopts a saliency map generation algorithm based on the UNet structure, using the intermediate feature map of the generated image as input to automatically detect edges, textures, and salient regions in the image. The initial threshold is set to 0.6. If the path score is high, the detection threshold is dynamically reduced to 0.4 to improve the sensitivity of detail recognition. The refinement intensity adjustment module is linked with the path score result. The higher the score, the higher the refinement intensity. The refinement parameter range is from the base value (2 pixels) to 8 pixels after the score is dynamically magnified. The pyramid convolutional refinement network includes three sets of convolutional channels of different scales (3×3, 5×5, 7×7). After inputting the key region, it enhances the local details layer by layer. Comparative experiments show that the peak signal-to-noise ratio (PSNR) of the refined region is improved by about 3.5dB. After the refined feature map is fused with the original image, it is directly passed to the next unit's discrimination stage.

[0075] Multi-scale fusion discrimination and feedback unit: connected to the local region recognition and refinement unit, it is used to perform parallel quality discrimination on the image generation results at the full-scale, local-scale and feature map scales, and to weight and fuse the discrimination results of each scale to form an overall quality feedback. The feedback is transmitted to the condition parsing and path scoring unit in real time for dynamic updating of path scoring parameters.

[0076] Multi-scale fusion discrimination and feedback unit embodiment: In this embodiment, the multi-scale fusion discrimination and feedback unit is set as a three-level quality discrimination structure to perform quality discrimination in parallel at the full image scale (input image size 512×512), the local scale (saliency detection output region, size 128x128), and the feature map scale (convolutional feature map, size 64×64). The full image scale adopts a quality evaluation module based on an improved VGG network, the local scale adopts a local discriminator, and the feature map scale performs feature quality evaluation through cosine similarity calculation. The scoring results of each scale are fused using the following weighted formula:

[0077] ,

[0078] in, To determine the overall image quality score, For local quality scores, The system calculates the feature map quality score; the fused quality score is sent back to the path scoring unit in real time, and the path scoring parameters are dynamically adjusted; if the fused score is lower than the preset threshold (e.g., 0.75), the system will automatically readjust the path and regenerate it until the score is qualified.

[0079] Image output and display unit: connected to the multi-scale fusion discrimination and feedback unit, used to output the final generated target image after the quality discrimination result reaches the preset threshold. The image output process must verify whether the quality discrimination score is qualified. If it is not qualified, it automatically returns to the condition parsing and path scoring unit to adjust the path scoring parameters and re-drive the image generation.

[0080] Image output and display unit embodiment:

[0081] In this embodiment, the image output and display unit is integrated into the user interface. This unit receives the final quality score result output by the multi-scale fusion discrimination unit in real time. When the score reaches or exceeds 0.75, image output and display are allowed. The image output format supports JPG and PNG formats, with adaptive resolution and a default output size of 512×512 pixels. It also displays a comparison image before and after local refinement, making it easy for users to observe the local quality improvement effect. If the score does not meet the quality standard, the system will automatically block the output permission, send the score result back to the path scoring unit, adjust the path parameters and regenerate, and allow image output only after the quality score is qualified, ensuring that the generated image meets the target quality requirements.

[0082] Among them, condition parsing and path scoring refer to the process by which this device receives natural language descriptions, reference image samples, and parameterized control vectors from user input. The system first performs semantic parsing, image feature extraction, and parameter association mapping on the input content. Through cross-modal association technology, different types of input are uniformly parsed into structured condition vectors. The generated condition vectors comprehensively describe the user's intent, reference sample features, and generation parameter constraints, and provide basic data support for subsequent path scoring and generation, ensuring full utilization and dynamic adaptation of input information. Based on the generated condition vectors, the system calls candidate paths from a preset image generation path library and scores them in real time based on path matching degree, multi-dimensional feature fit, and generation target deviation. The scoring results are not only used for current path optimization but also dynamically fed back to adjust the generation parameters of subsequent paths, realizing a real-time closed loop of path scoring and generation control. The scoring process references a weighted comprehensive function, and the weight factors are dynamically adjusted according to the condition vectors to ensure that the path selection meets user needs and the generation effect is stable and reliable.

[0083] The formula for expressing path scoring is:

[0084] ,

[0085] In the formula, path The scoring results (dimensionless) : Input the matching degree weight coefficient (range 0~1), Input conditions and path The matching degree (0~1, calculated using cosine similarity) Historical generation quality weighting coefficient (range 0~1), :path Historical image quality score (structural similarity index, SSIM, range 0~1), Generate complexity weight coefficients (range 0~1). :path The generation complexity (unit: FLOPs, floating-point computation).

[0086] The dynamic path selection and generation unit, connected to the condition parsing and path scoring unit, selects the optimal image generation path from preset paths based on the scoring results and configures the structure and layout of the generation network in real time based on this path. The layer depth, number of channels, and feature transfer method of the generation network are dynamically adjusted according to the scoring results, making the generation process targeted. The scoring results serve as dynamic configuration parameter inputs, ensuring that the selected path highly matches the user input conditions, improving the accuracy and diversity of image generation. Guided by the selected image generation path, the system generates multi-scale image features layer by layer along the path, driving the dynamic adjustment of path generation nodes, convolution kernel parameters, and skip connection structures in real time. The adjustment of convolution kernel parameters is controlled by a parameter mapping function based on scoring weights, ensuring that the convolution kernel size, stride, and channel response characteristics are highly consistent with the current path score, dynamically responding to changes in input conditions. This path control mechanism effectively improves image detail preservation and structural coherence.

[0087] The formula for dynamically adjusting the convolution kernel parameters is:

[0088] ,

[0089] In the formula, The kernel size (in pixels) used in the current generated path. : Basic convolutional kernel size (unit: pixels) Convolution kernel adjustment factor (unit: pixels) The selected path score is consistent with the calculation result of Unit 1.

[0090] Among them, local region identification and refinement refers to the connection between this unit and the dynamic path selection and generation unit to monitor key detail regions in real time during the image generation process; the system dynamically identifies key regions that affect image quality based on the current path score results and adjusts the local refinement intensity in real time; the local refinement intensity is dynamically calculated through an association adjustment function based on the score feedback, and the change in refinement intensity is linked to the path score weight to ensure that key regions obtain higher resolution detail processing and improve the overall image quality;

[0091] The formula for adjusting the local refinement intensity is:

[0092] ,

[0093] In the formula, Current local refinement intensity parameters (unit: pixels), Base refinement intensity (unit: pixels), Refine intensity adjustment factor (unit: pixels), The selected path scoring result (consistent with the calculation result of Unit 1); for key areas, the system calls the pyramid convolutional network to refine layer by layer, generating multi-scale high-resolution features; during the refinement process, the local convolutional kernel parameters and stride are dynamically adjusted under the drive of refinement intensity, and the refinement result is fused with the original image features in real time to ensure that the details are highly consistent with the overall structure; the refinement intensity adjustment relies on the hierarchical mapping strategy of the scoring result, so that the refinement degree and path selection are dynamically matched, improving the local detail restoration and generation continuity of the image.

[0094] The multi-scale fusion discrimination and feedback unit connects this unit with the local region recognition and refinement unit to perform parallel quality discrimination at the full-scale, local key-scale, and feature map scales during image generation. The system evaluates sharpness, structural consistency, and detail reproduction at each scale to form multi-dimensional quality indicators. The quality discrimination results at each scale are weighted and fused based on a weight adjustment function. The fusion result constructs a global quality feedback to ensure comprehensive and accurate quality evaluation with real-time response capabilities. The overall quality feedback result after fusion is transmitted to the condition parsing and path scoring unit in real time as an important input for the dynamic adjustment of path scoring parameters. The system gradually optimizes the path scoring parameters through a quality feedback mapping function, ensuring that the path selection and generation strategy continuously adapt to changes in image quality. The quality scoring fusion process references a weighted integral mechanism, with the fusion weights adjusted in real time as the image generation process progresses, ensuring the feedback process is timely and targeted, and improving the adaptive capability of the generated path.

[0095] The formula for expressing the quality score fusion is:

[0096] ,

[0097] In the formula, Overall image quality score after fusion (unit: dimensionless, range 0~1) :No. The rating weights of the scale (weight coefficients, summing to 1), :No. Quality score of the scale (unit: dimensionless, range 0~1), Total number of scales to be judged (e.g., 3 scales).

[0098] The image output and display unit refers to the connection between this unit and the multi-scale fusion discrimination and feedback unit. After the image is generated, the fusion quality score is first judged based on the preset quality threshold. The system uses a quality verification function to check the current image item by item to ensure that the output image meets the quality requirements in terms of sharpness, structural consistency and detail integrity. The image can only enter the final output process when the discrimination score meets the output conditions, so as to ensure that the image result meets the expected standard.

[0099] The formula for determining image output is:

[0100] ,

[0101] In the formula, Output: whether the image is allowed to be output (Pass: allowed; Reject: not allowed). Integration quality score (from Unit 4) A preset quality threshold (range 0~1) is set. When the image quality judgment result is lower than the set threshold, the system automatically triggers the path adjustment process and transmits the current quality feedback back to the condition parsing and path scoring unit in real time. The path scoring parameters are dynamically updated according to the quality deviation, the optimal image generation path is reselected, and a new round of image generation process is driven. The path adjustment is iterated in multiple rounds according to the scoring correction mapping relationship to ensure continuous optimization of the generation path and finally output a high-quality image, forming a closed-loop control.

[0102] This invention also provides a deep learning-based image generation method based on the above-mentioned units, the specific steps of which are as follows:

[0103] S1, Condition Parsing and Path Scoring: Receives natural language description, reference image and parameter vector input by the user, parses them into structured condition vector, and scores the preset image generation path library in real time based on the path scoring algorithm. The scoring result is used as the parameter input for subsequent path generation.

[0104] S2, Path Selection and Local Refinement Generation: Select the optimal generation path based on the results of condition parsing and path scoring, dynamically configure the generation network structure based on the scoring results, generate multi-scale image features layer by layer along the selected path, detect key detail regions in real time, and adjust the local refinement intensity synchronously based on the path score to perform high-resolution refinement on key regions.

[0105] S3, Conditional Analysis and Path Scoring: The quality judgment of the path selection and local refinement generated image is performed in parallel at the full-scale, local-scale and feature map scales. The judgment results of each scale are fused to form an overall quality feedback. The feedback results are transmitted to step one in real time to adjust the path scoring parameters. If the judgment score is lower than the quality threshold, it automatically returns to step one to re-execute. If the judgment score meets the requirements, the final generated image is output.

[0106] In S1, condition parsing and path scoring refer to the system receiving natural language descriptions, reference image samples, and parameter control vectors from user input. Semantic features are extracted from the natural language, image content is analyzed from the reference images, and the parameter vectors are numerically normalized. Through cross-modal parsing and association mapping, the system transforms multi-source inputs into unified structured condition vectors. These condition vectors fully represent user intent, reference sample features, and generation control requirements, providing accurate input for subsequent path scoring and generation processes. The system uses the parsed structured condition vectors to score candidate paths in a pre-defined image generation path library in real time. The path scoring process comprehensively considers path feature matching, parameter adaptability, and historical generation effects, dynamically calculating the optimal score for each path. The scoring results are used as parameter inputs for path generation in real time, driving the dynamic selection and structural configuration of subsequent image generation paths, ensuring a high degree of matching between the selected path and user input, and achieving efficient and accurate image generation control.

[0107] In S2, path selection and local refinement generation refer to the system optimizing the image generation path based on condition parsing and path scoring results, and dynamically configuring the hierarchical structure, channel parameters, and connection methods of the generation network based on the scoring results. Multi-scale image features are generated layer by layer along the selected path to ensure a reasonable generation structure and that the generation path is highly consistent with the user input, thereby improving the overall image quality and generation efficiency. During the layer-by-layer image generation process, the system detects key detail areas in real time, focuses on identifying important image content, and adjusts the local refinement intensity synchronously based on the path scoring results. A multi-scale high-resolution refinement strategy is used to refine key areas, ensuring rich details and structural integrity, thereby improving the local image quality and visual realism.

[0108] In S3, conditional parsing and path scoring refer to the system's parallel quality judgment of the image generated by path selection and local refinement at the full-image scale, local key region scale, and feature map scale. The quality indicators at each scale are fused in real time to form a comprehensive overall quality feedback. This feedback result is synchronously transmitted to the path scoring parameter adjustment unit to dynamically optimize the path selection and generation parameters, ensuring accurate and timely quality evaluation. The system performs a threshold judgment on the fused quality feedback result. If the quality score is lower than the preset standard, it immediately returns to the path scoring step, reselects the generation path, and executes the image generation process. If the judgment score meets the quality requirements, the final generated target image is output, ensuring that the generated result meets the user's expectations and forming an efficient closed-loop quality control process.

[0109] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0110] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An image generation device based on deep learning, characterized in that: The device consists of a condition parsing and path scoring unit, a dynamic path selection and generation unit, a local region recognition and refinement unit, a multi-scale fusion discrimination and feedback unit, and an image output and display unit. Condition parsing and path scoring unit: This unit receives a natural language description, reference image samples, and parameterized control vectors input by the user; parses and generates condition vectors; and scores candidate paths in a preset image generation path library in real time based on these condition vectors, outputting the scoring results. The real-time scoring is a path scoring of the candidate paths, and the path score is a weighted score that comprehensively considers the matching degree between the condition vectors and the candidate paths, the historical generation quality scores of the candidate paths, and the generation complexity of the candidate paths. Dynamic path selection and generation unit: used to select the optimal image generation path according to the scoring result, dynamically configure the generation network structure based on the scoring result, the dynamic configuration includes at least adjusting the convolution kernel parameters or the hierarchical structure of the generation network according to the scoring components corresponding to the generation complexity, and generating multi-scale image features layer by layer along the selected path, the scoring result drives the dynamic adjustment of path generation nodes, convolution kernel parameters and jump structure in real time; Local region recognition and thinning unit: used to detect key detail regions in real time during image generation, and dynamically adjust the local thinning intensity based on path scoring results. It uses a pyramid convolutional network to perform high-resolution thinning of key regions, and the thinning results are fused with the original image features in real time. Multi-scale fusion discrimination and feedback unit: It is used to perform parallel quality discrimination on the image generation results at the full-scale, local-scale and feature map scales, and to weight and fuse the discrimination results of each scale to form an overall quality feedback. The feedback is transmitted to the condition parsing and path scoring unit in real time for dynamic updating of path scoring parameters. Image output and display unit: used to output the final generated target image after the quality discrimination result reaches the preset threshold. The image output process must verify whether the quality discrimination score is qualified. If it is not qualified, it automatically returns to the condition parsing and path scoring unit to adjust the path scoring parameters and re-drive the image generation.

2. The image generation device based on deep learning according to claim 1, characterized in that: The condition parsing and path scoring unit includes: (1) User input parsing and condition vector construction: Natural language description, reference image sample and parameterized control vector are used to receive user input. The system first performs semantic parsing, image feature extraction and parameter association mapping on the input content. Through cross-modal association technology, different types of input are uniformly parsed into structured condition vectors. The generated condition vectors comprehensively describe the user intent, reference sample features and generation parameter constraints. (2) Real-time path scoring and dynamic generation control: Based on the generated condition vector, candidate paths are called from the preset image generation path library, and real-time scoring is performed by path matching degree, multi-dimensional feature fit degree and generation target deviation; the scoring results are not only used for current path optimization, but also dynamically fed back to adjust the subsequent path generation parameters, realizing real-time closed loop of path scoring and generation control; the scoring process refers to the weighted comprehensive function, and the weight factor is dynamically adjusted according to the condition vector.

3. The image generation device based on deep learning according to claim 1, characterized in that: The dynamic path selection and generation unit includes: (1) Optimal path selection and dynamic configuration of the generation network: The optimal image generation path is selected from the preset path based on the scoring results, and the structure layout of the generation network is configured in real time based on the path; the layer depth, number of channels and feature transfer method of the generation network are dynamically adapted and adjusted according to the scoring results, so that the generation process is targeted; the scoring results are used as dynamic configuration parameters to ensure that the selected path is highly matched with the user input conditions. (2) Path-driven feature generation and convolutional structure adjustment: Under the guidance of the selected image generation path, the system generates multi-scale image features layer by layer along the path, and drives the dynamic adjustment of path generation nodes, convolutional kernel parameters and skip connection structure in real time; the adjustment of convolutional kernel parameters is controlled by a parameter mapping function based on scoring weights to ensure that the convolutional kernel size, stride and channel response characteristics are highly consistent with the current path score and dynamically respond to changes in input conditions.

4. The image generation device based on deep learning according to claim 1, characterized in that: The local region identification and refinement unit includes: (1) Detection of key detail regions and local thinning control: Real-time monitoring of key detail regions during image generation; Based on the current path score results, the system dynamically identifies key regions that affect image quality and adjusts the local thinning intensity in real time; The local thinning intensity is dynamically calculated through an association adjustment function based on the score feedback, and the change in thinning intensity is linked to the path score weight. (2) Pyramid refinement and feature fusion mechanism: For key areas, the system calls the pyramid convolutional network to refine layer by layer and generate multi-scale high-resolution features; during the refinement process, the local convolutional kernel parameters and stride are dynamically adjusted under the drive of refinement intensity, and the refinement result is fused with the original image features in real time; the refinement intensity adjustment relies on the hierarchical mapping strategy of the scoring results to dynamically match the refinement degree with the path selection.

5. The image generation device based on deep learning according to claim 1, characterized in that: The multi-scale fusion discrimination and feedback unit includes: (1) Parallel quality discrimination and scoring feedback at all scales: Parallel quality discrimination is performed at the full-scale, local key scale and feature map scale in the image generation process; the system evaluates the sharpness, structural consistency and detail reproduction at each scale respectively to form multi-dimensional quality indicators; the quality discrimination results at each scale are weighted and fused based on the weight adjustment function, and the fusion results are used to construct global quality feedback. (2) Quality feedback drives dynamic update of path scoring parameters: The overall quality feedback result after fusion will be transmitted to the condition parsing and path scoring unit in real time as an important input for dynamic adjustment of path scoring parameters. The system optimizes the path scoring parameters step by step through the quality feedback mapping function so that the path selection and generation strategy continuously fits the changes in image quality. The quality scoring fusion process refers to the weighted integral mechanism, and the fusion weight is adjusted in real time with the image generation process.

6. The image generation device based on deep learning according to claim 1, characterized in that: The image output and display unit includes: (1) Image output judgment and quality verification process: After the image is generated, the fusion quality score is judged based on the preset quality threshold. The system uses the quality verification function to check the current image item by item to ensure that the output image meets the quality requirements in terms of clarity, structural consistency and detail integrity. The image can only enter the final output process when the judgment score meets the output conditions. (2) Adaptive path adjustment when quality is not up to standard: When the image quality judgment result is lower than the set threshold, the system automatically triggers the path adjustment process and transmits the current quality feedback back to the condition analysis and path scoring unit in real time; the path scoring parameters are dynamically updated according to the quality deviation, the optimal image generation path is reselected, and a new round of image generation process is driven; the path adjustment is iterated in multiple rounds according to the scoring correction mapping relationship.

7. A deep learning-based image generation method, characterized in that: The method includes the image generation apparatus based on deep learning as described in any one of claims 1 to 6, and the specific steps of the method are as follows: S1, Condition Parsing and Path Scoring: Receives natural language description, reference image and parameter vector input by the user, parses them into structured condition vectors, performs real-time scoring on the preset image generation path library based on the path scoring algorithm, and outputs the scoring results; S2, Path Selection and Local Refinement Generation: The optimal generation path is selected based on the results of condition parsing and path scoring. The generation network structure is dynamically configured based on the scoring results. Multi-scale image features are generated layer by layer along the selected path. Key detail regions are detected in real time. The local refinement intensity is adjusted synchronously based on the path score to perform high-resolution refinement on key regions. S3, Multi-scale quality discrimination and fusion feedback: Quality discrimination is performed in parallel at the full-scale, local-scale, and feature map scales of the image generated by path selection and local refinement. The discrimination results of each scale are fused to form an overall quality feedback. The feedback results are transmitted to step one in real time to adjust the path scoring parameters. If the discrimination score is lower than the quality threshold, it automatically returns to step one to re-execute. If the discrimination score meets the requirements, the final generated image is output.

8. The image generation method based on deep learning according to claim 7, characterized in that: The specific steps of condition parsing and path scoring in S1 are as follows: S11, User Input Parsing and Condition Vector Construction: The system receives the user's input natural language description, reference image sample, and parameter control vector. Semantic features are extracted from the natural language, image content is analyzed from the reference image, and numerical normalization is performed on the parameter vector. Through cross-modal parsing and association mapping, the system transforms multi-source input into a unified structured condition vector. The condition vector fully represents the user's intent, reference sample features, and generation control requirements. S12, Path library scoring and parameter input dynamic configuration: The system uses the structured condition vector obtained by parsing to score the candidate paths in the preset image generation path library in real time; The path scoring process comprehensively considers path feature matching degree, parameter adaptability and historical generation effect, and dynamically calculates the optimal score of each path; the scoring results are used as parameter input for path generation in real time, driving the dynamic selection and structural configuration of subsequent image generation paths.

9. The image generation method based on deep learning according to claim 7, characterized in that: The specific steps for path selection and local refinement generation in S2 are as follows: S21, Optimal Path Selection and Dynamic Configuration of Generation Network: Based on the condition parsing and path scoring results, the system optimizes the image generation path and dynamically configures the hierarchical structure, channel parameters, and connection methods of the generation network based on the scoring results; multi-scale image features are generated layer by layer along the selected path; S22, Key Detail Detection and Local High-Resolution Refinement: During the layer-by-layer generation of the image, the system detects key detail regions in real time, focuses on identifying important image content, and adjusts the local refinement intensity synchronously based on the path scoring results. It also employs a multi-scale high-resolution refinement strategy to perform fine processing on key regions.

10. The image generation method based on deep learning according to claim 7, characterized in that: The specific steps of multi-scale quality discrimination and fusion feedback in S3 are as follows: S31, Multi-scale quality discrimination and fusion feedback: The system performs quality discrimination on the image generated by path selection and local refinement in parallel at the full image scale, the local key region scale and the feature map scale; the quality indicators of each scale are fused in real time to form a comprehensive overall quality feedback. The feedback result is simultaneously transmitted to the path scoring parameter adjustment unit; S32, Quality Threshold Verification and Generation Process Control: The system performs threshold judgment on the quality feedback results after fusion. If the quality score is lower than the preset standard, it immediately returns to the path scoring step, reselects the generation path, and executes the image generation process. If the discrimination score meets the quality requirements, the final generated target image is output.

Images