Image generation method and electronic device

Abstract

The application discloses an image generation method and an electronic device. The method comprises the following steps: acquiring an input image; acquiring at least one first candidate sample image matched with the input image from a sample image set; the at least one first candidate sample image all has annotation information, and the annotation information is used for distinguishing at least two regions of the first candidate sample image; wherein the first candidate sample image contains a sample image matched with a second candidate sample image in the sample image set; the second candidate sample image is matched with the input image and does not have annotation information; based on the at least one first candidate sample image, the style of the at least two regions of the input image is enhanced, and at least two regions after enhancement processing are obtained; and based on the at least two regions after enhancement processing, a target image is generated.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an image generation method and electronic device. Background Technology

[0002] Currently, when there is a style shift between synthetic and real images, global, random style transfer or blending methods are often used to correct the synthetic image. However, this method is prone to causing semantic distortion of the synthetic image or introducing invalid visual noise into the synthetic image, resulting in poor accuracy of style shift correction. Summary of the Invention

[0003] The technical solution provided in this application is as follows:

[0004] The first aspect of this application provides an image generation method, comprising:

[0005] Obtain the input image;

[0006] At least one first candidate sample image matching the input image is obtained from the sample image set; each of the at least one first candidate sample image has annotation information, which is used to distinguish at least two regions of the first candidate sample image; wherein, the first candidate sample image includes a sample image matching a second candidate sample image from the sample image set; the second candidate sample image matches the input image but does not have annotation information;

[0007] Based on the at least one first candidate sample image, the style of at least two regions of the input image is enhanced to obtain at least two enhanced regions;

[0008] A target image is generated based on at least two regions after the enhancement process.

[0009] In one possible implementation, obtaining at least one first candidate sample image matching the input image from the sample image set includes:

[0010] Obtain a first-class first-candidate sample image and / or a second-candidate sample image that matches the input image from a set of sample images; the first-class first-candidate sample image has annotation information; the second-candidate sample image does not have annotation information; the annotation information is used to distinguish at least two regions of the corresponding candidate sample image;

[0011] Once the existence of the second candidate sample image is determined, a second type of first candidate sample image matching the second candidate sample image is obtained from the sample image set; the second type of first candidate sample image has annotation information.

[0012] In one possible implementation, the step of enhancing the style of at least two regions of the input image based on the at least one first candidate sample image to obtain at least two enhanced regions includes:

[0013] Based on the annotation information of the first candidate sample image of the first type and the first candidate sample image of the second type respectively, determine the style parameters corresponding to at least two regions of the first candidate sample image of the first type and the first candidate sample image of the second type respectively;

[0014] Based on the style parameters corresponding to at least two regions of the second type of first candidate sample image, and the similarity between the second type of first candidate sample image and the second candidate sample image, the style parameters corresponding to at least two regions of the second candidate sample image are determined.

[0015] Based on the style parameters corresponding to at least two regions of each of the first candidate sample image and the second candidate sample image, and the similarity between each of the first candidate sample image and the second candidate sample image and the input image, the style parameters corresponding to at least two regions of the input image are determined.

[0016] Based on the style parameters corresponding to at least two regions of the input image, enhancement processing is performed on at least two regions of the input image to obtain at least two enhanced regions.

[0017] In one possible implementation, there are multiple first candidate sample images of the second type; determining the style parameters corresponding to at least two regions of the second candidate sample image based on the style parameters corresponding to at least two regions of the second candidate sample image and the similarity between the second candidate sample image and the second candidate sample image includes:

[0018] Based on the similarity between each second-class first candidate sample image and the second candidate sample image, a first weight factor is determined for each second-class first candidate sample image; the first weight factor characterizes the relative similarity between the second candidate sample image and the second-class first candidate sample image.

[0019] Based on the first weighting factor, the style parameters corresponding to at least two regions of each of the second type of first candidate sample images are fused to obtain the style parameters corresponding to at least two regions of the second candidate sample image.

[0020] In one possible implementation, determining the style parameters corresponding to at least two regions of the input image based on the style parameters corresponding to at least two regions of each of the first-class first candidate sample image and the second candidate sample image, and the similarity between each of the first-class first candidate sample image and the second candidate sample image and the input image, includes:

[0021] Based on the similarity between the first candidate sample image and the second candidate sample image and the input image, a second weighting factor and a third weighting factor are determined for the first candidate sample image and the second candidate sample image, respectively; the second weighting factor represents the relative similarity between the first candidate sample image and the input image; the third weighting factor represents the relative similarity between the second candidate sample image and the input image.

[0022] Based on the second weighting factor and the third weighting factor, the style parameters corresponding to at least two regions of the first candidate sample image and the second candidate sample image are fused to obtain the style parameters corresponding to at least two regions of the input image.

[0023] In one possible implementation, the enhancement processing of at least two regions in the input image based on style parameters corresponding to each of the at least two regions of the input image to obtain at least two enhanced regions includes:

[0024] Based on the style parameters corresponding to at least two regions of the input image, style normalization processing is performed on at least two regions of the input image.

[0025] Based on the style parameters corresponding to at least two regions of the input image, style reconstruction processing is performed on the at least two regions after normalization to obtain at least two regions after enhancement processing.

[0026] In one possible implementation, generating the target image based on at least two regions after the enhancement process includes:

[0027] Obtain the annotation information of the input image; the annotation information of the input image includes at least two regions used to distinguish the input image;

[0028] Based on the annotation information of the input image, denoising processing is performed on at least two regions after the enhancement process;

[0029] At least two regions after denoising are fused to obtain the target image.

[0030] In one possible implementation, obtaining a second type of first candidate sample image that matches the second candidate sample image from the sample image set includes:

[0031] Use the second candidate sample image as the current query image;

[0032] Query at least one third candidate sample image from the set of sample images that is similar to the currently queried image;

[0033] If none of the at least one third candidate sample image has annotation information, the current query image is updated based on the at least one third candidate sample image, and the step of selecting at least one third candidate sample image similar to the current query image from the sample image set is continued until the at least one third candidate sample image contains a second type of first candidate sample image or the set number of queries is reached.

[0034] In one possible implementation, updating the current query image based on the at least one third candidate sample image includes any one of the following:

[0035] The third candidate image with the highest similarity to the current query image among the at least one third candidate image is taken as the current query image;

[0036] Based on the similarity between each third candidate sample image in the at least one third candidate sample image and the current query image, the third candidate sample images are fused together, and the fused image is used as the current query image.

[0037] In another aspect, this application provides an electronic device, comprising:

[0038] Memory is used to store computer programs;

[0039] The processor is used to execute the computer program to enable the electronic device to perform the following method steps:

[0040] Obtain the input image;

[0041] At least one first candidate sample image matching the input image is obtained from the sample image set; each of the at least one first candidate sample image has annotation information, which is used to distinguish at least two regions of the first candidate sample image; wherein, the first candidate sample image includes a sample image matching a second candidate sample image from the sample image set; the second candidate sample image matches the input image but does not have annotation information;

[0042] Based on the at least one first candidate sample image, the style of at least two regions of the input image is enhanced to obtain at least two enhanced regions;

[0043] A target image is generated based on at least two regions after the enhancement process. Attached Figure Description

[0044] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0045] Figure 1 This is a flowchart illustrating an image generation method provided in Embodiment 1 of this application;

[0046] Figure 2 This is a schematic flowchart of an image generation method provided in Embodiment 2 of this application;

[0047] Figure 3 This application provides a schematic diagram of a process for determining a first candidate sample image;

[0048] Figure 4 This is a flowchart illustrating an image generation method provided in Embodiment 3 of this application;

[0049] Figure 5 This is a flowchart illustrating an image generation method provided in Embodiment 4 of this application;

[0050] Figure 6 This is a flowchart illustrating an image generation method provided in Embodiment 5 of this application;

[0051] Figure 7 This is a flowchart illustrating an image generation method provided in Embodiment 6 of this application;

[0052] Figure 8 This is a flowchart illustrating an image generation method provided in Embodiment 7 of this application;

[0053] Figure 9 This is a schematic diagram illustrating an implementation scenario of an image generation method provided in this application.

[0054] Figure 10 This is a schematic diagram of the structure of an electronic device provided in this application. Detailed Implementation

[0055] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.

[0056] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.

[0057] The terms "first," "second," etc., used in this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of units is not necessarily limited to those units, but may include other units not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0058] Reference Figure 1 This is a flowchart illustrating an image generation method provided in Embodiment 1 of this application, as shown below. Figure 1 As shown, the method may include, but is not limited to, the following steps:

[0059] Step S101: Obtain the input image.

[0060] In practical applications, high-quality labeled data is often costly to obtain and has a limited sample size. Models trained directly on a small amount of real data have insufficient generalization ability. Therefore, input images can be introduced as source data for style transfer and sample augmentation to generate high-quality training samples that are adapted to the distribution of the target scene and meet the data requirements for iterative training of the model.

[0061] The input image may include, but is not limited to, two types: synthetic images and real-world scene images.

[0062] Among them, synthetic images can be generated through 3D modeling and rendering, or through generative artificial intelligence models (such as diffusion models, generative adversarial networks, etc.), or through game engines (such as Unity, Unreal Engine, etc.) for scene building and image rendering.

[0063] Real-world scene images can be captured using image acquisition devices. These devices may include, but are not limited to, industrial cameras, surveillance cameras, ordinary digital cameras, and mobile phone cameras. The scenarios in which the images are captured may include, but are not limited to, various application scenarios such as industrial inspection, autonomous driving, security monitoring, medical imaging, and agricultural inspection.

[0064] Step S102: Obtain at least one first candidate sample image that matches the input image from the sample image set; the at least one first candidate sample image has annotation information, which is used to distinguish at least two regions of the first candidate sample image; wherein, the first candidate sample image includes a sample image in the sample image set that matches the second candidate sample image; the second candidate sample image matches the input image but does not have annotation information.

[0065] The sample image set may include sample images of real scenes. These sample images may cover a variety of typical visual styles under the target application scenarios (such as industrial inspection, autonomous driving, security monitoring, medical imaging, agricultural inspection, etc.), including but not limited to style dimensions such as different lighting conditions, different material textures, different color distributions, and different imaging device features.

[0066] The sample image set can be constructed in a flexible and diverse manner, including but not limited to:

[0067] Each set of sample images corresponds to a specific visual style for a particular application scenario. For example, a separate set of sample images is constructed for the imaging style of metal workpieces under cool fluorescent lighting conditions in industrial inspection scenarios; another set is constructed for the imaging style of ceramic insulators under strong direct sunlight conditions in power line inspection scenarios; and yet another set is constructed for the imaging style of ceramic insulators under cloudy diffused light conditions in the same power line inspection scenario. In other words, each style corresponds to an independent set of sample images, and different styles are isolated from each other. The corresponding set of sample images can be accurately matched and retrieved based on the specific scene and style attributes of the input image, ensuring a high degree of specificity in style reference.

[0068] A single sample image set covers multiple typical visual styles within the same application scenario. For example, a sample image set can be constructed for power line inspection scenarios. This set simultaneously includes sample images of ceramic insulators under various lighting conditions, such as strong sunlight on sunny days, diffused light on cloudy days, low-angle light at dusk, and infrared supplemental lighting at night. It also includes sample images of different types of insulators (such as disc suspension insulators, pin insulators, and rod insulators) under different surface conditions (such as clean surfaces, contaminated surfaces, and icy surfaces), as well as sample images with different color distributions and imaging equipment characteristics. This approach is suitable for implementation requirements where there are many style variations within the same scenario, and multiple style references need to be obtained in a single matching operation.

[0069] A single set of sample images can cover the same visual style across different application scenarios. For example, a set of sample images with a cool-toned industrial style can be constructed. This set could include images of metal workpieces under cool lighting in industrial inspection scenarios, images of ceramic insulators under diffused light on cloudy days in power line inspection scenarios, images of road signs under cool-toned light in the early morning in autonomous driving scenarios, and pedestrian images in infrared mode at night in security monitoring scenarios—sample images that all exhibit cool-toned characteristics across different application scenarios. This approach is suitable for implementation needs that require the reuse of a certain general style feature across scenarios.

[0070] A single sample image set is not limited to a specific scene or style. It includes sample images from various application scenarios (such as industrial inspection, power line inspection, autonomous driving, security monitoring, medical imaging, and agricultural inspection) and various visual styles (such as different lighting conditions, different material textures, different color distributions, and different imaging device features), forming a comprehensive style reference dataset with strong general applicability. This approach is suitable for general model training scenarios where there are no strict limitations on scenes and styles and the goal is to maximize sample coverage.

[0071] For example, this sample image set can simultaneously include: images of scratches and defects on the surface of metal workpieces in industrial inspection scenarios; images of surface contamination and damage of ceramic insulators under different lighting and weather conditions in power line inspection scenarios; images of road signs during day and night transitions in autonomous driving scenarios; pedestrian images in nighttime infrared mode in security monitoring scenarios; color images of crop leaves at different growth stages in agricultural inspection scenarios; and X-ray images acquired by different devices in medical imaging scenarios. By uniformly including these cross-scenario and cross-style sample images in the same set, a richer style reference dimension can be obtained when performing style matching, improving the generalization ability and adaptability of style enhancement processing.

[0072] In the actual data construction process, regional annotation of sample images requires a large amount of manual labor or relies on high-precision intelligent annotation tools, making it difficult to annotate all samples one by one. However, unannotated sample images also carry rich real-world scene style features, and discarding them directly would result in a waste of data resources. Therefore, in this embodiment, sample images with annotation information (i.e., annotated sample images) and sample images without annotation information (i.e., unannotated sample images) can be retained simultaneously in the sample image set.

[0073] Labeling information can be generated manually or through intelligent labeling. Manual labeling involves labelers using professional labeling tools (such as LabelMe, CVAT, etc.) to segment and categorize images. Intelligent labeling utilizes pre-trained semantic segmentation models or large-scale model-assisted labeling tools to automatically generate segmentation results, which are then manually verified.

[0074] The specific forms of the annotation information include, but are not limited to: segmentation masks, pixel-level category label maps, bounding boxes, etc. Taking a segmentation mask as an example, the annotation information exists in the form of a mask image of the same size as the original image. The value of each pixel in the mask image corresponds to the semantic region category to which it belongs, thereby achieving pixel-level accurate division of each region of the image.

[0075] At least two regions can include, but are not limited to, a foreground region and a background region. The foreground region corresponds to the main target object in the image (such as an insulator), and the background region corresponds to the environmental scene (such as the sky). In addition, at least two regions can also be represented by other division methods, such as a subject region and a non-subject region. The subject region refers to the visually significant or semantically core part of the image, and the non-subject region refers to the rest of the image. This can also achieve regionalized style processing.

[0076] The first candidate sample image is a real scene image within the sample image set that matches the features of the input image and carries complete annotation information. Based on its annotation information, at least two regions within the first candidate sample image can be accurately divided, and the regional style of each region can be determined independently.

[0077] The second candidate sample image is a real scene image within the sample image set that matches the features of the input image but does not carry annotation information. Due to the lack of annotation basis for region division, it cannot independently complete multi-region segmentation, cannot independently distinguish the style differences of different regions, and cannot be directly used for region-specific style enhancement operations.

[0078] To fully utilize the style carried by these unlabeled samples, after selecting a second candidate sample image that matches the input image from the sample image set, labeled sample images that match the features of the second candidate sample image can be further retrieved from the sample image set and included in the set of the first candidate sample images, thus achieving effective reuse of the style of the unlabeled sample images.

[0079] Step S103: Based on the at least one first candidate sample image, perform style enhancement processing on at least two regions of the input image to obtain at least two enhanced regions.

[0080] In real-world images, different areas possess independent visual style attributes. Taking a power line inspection scene as an example, the main area of ​​the ceramic insulator and the background sky area have inherent differences in style dimensions such as lighting performance, texture, color distribution, and imaging characteristics. The style of the main insulator area is reflected in the texture and color performance of the real ceramic material under the current lighting, while the style of the background sky area is reflected in the color temperature and hue of the real sky and the color distribution of the clouds. The real style distributions of the two are independent of each other and cannot be mixed.

[0081] Based on the aforementioned regional style independence, this embodiment can perform independent style enhancement processing on at least two regions obtained by dividing the input image, respectively, based on the style of the corresponding region in the at least one first candidate sample image, so that each region of the input image conforms to its inherent style distribution pattern in the real scene.

[0082] Through the above-mentioned decoupled style optimization by region, the style enhancement of each region is independent and does not interfere with each other. It not only fully preserves the semantic structure and feature information of the core target of the image, but also accurately matches the real scene style corresponding to each region. It avoids the defects of cross-regional style interference, semantic distortion and invalid noise introduction caused by the global unified correction method, and greatly improves the accuracy and naturalness of image style shift correction.

[0083] Step S104: Generate a target image based on at least two regions after the enhancement process.

[0084] After obtaining at least two enhanced regions, the enhanced regions can be fused and stitched together according to their original spatial relationships in the input image to generate the target image.

[0085] At the boundaries of regions, further smoothing processes can be performed, such as using Poisson blending or gradient feathering, to eliminate visual defects such as color jumps, brightness discontinuities, or texture mismatches that may occur at the boundaries of region splicing, ensuring that the target image appears natural and coherent with no obvious splicing marks.

[0086] Taking a power line inspection scenario as an example, after step S103, the style-enhanced insulator main area and background sky area have been obtained respectively. In step S104, the two enhanced areas can be merged and stitched according to their original spatial layout in the input image, and a smooth transition is made at the junction of the insulator edge and the sky in a gradient manner, finally generating a high-quality target image that is completely consistent with the original input image in terms of content semantics and closely resembles the real scene image in terms of style.

[0087] In this embodiment, after selecting a second candidate sample image that matches the input image, annotated sample images that match the features of the second candidate sample image are further retrieved and included in the set of first candidate sample images. This allows for full reuse of the real scene style carried by the second candidate sample image without increasing the additional region-level annotation cost. It effectively avoids data waste caused by the idleness of unlabeled samples, thereby maximizing the use of the style of the sample images in the sample image set and significantly improving the style reference richness and robustness of style enhancement processing.

[0088] Based on this, at least two regions are distinguished using the annotation information in the expanded first candidate sample image. Independent style enhancement processing is then performed on the at least two regions obtained from the input image based on the style of the corresponding region in the first candidate sample image. This enables decoupled style optimization of each region, ensuring that each region of the input image conforms to its inherent style distribution pattern in the real scene. This effectively avoids the defects of cross-region style interference, semantic distortion, and the introduction of invalid visual noise caused by global correction methods, and maintains the coherence and visual realism of the overall semantic structure of the image.

[0089] In summary, this embodiment, through the synergistic effect of effective reuse of unlabeled sample styles and regional independent enhancement mechanisms, significantly reduces data construction costs while generating high-quality target images that fully preserve the original information of the input image in terms of semantic content and closely resemble the distribution of real scenes in terms of style representation. This can effectively meet the model training requirements for high-quality style sample images.

[0090] As another optional embodiment of this application, refer to Figure 2 This is a flowchart illustrating an image generation method provided in Embodiment 2 of this application. This embodiment is mainly an implementation of step S102 in Embodiment 1, such as... Figure 2 As shown, it may include, but is not limited to:

[0091] Step S1021: Obtain a first type of first candidate sample image and / or a second candidate sample image that matches the input image from the sample image set; the first type of first candidate sample image has annotation information; the second candidate sample image does not have annotation information; the annotation information is used to distinguish at least two regions of the corresponding candidate sample image.

[0092] In this embodiment, the similarity between each sample image in the sample image set and the input image can be determined using the following similarity calculation formula:

[0093]

[0094] in, This represents the similarity between the j-th sample image in the sample image set and the input image. Input image; Let j be the j-th sample image in the sample image set; is a feature extraction function that can be used to extract semantic features of an image. Semantic features can be used to characterize semantic information such as scene type, target subject, structural layout, and content context of an image; sim() is a similarity measurement function used to calculate the degree of matching between two images in the feature space.

[0095] In this embodiment, the calculated similarity can be sorted from high to low, and the top K sample images can be selected.

[0096] In this embodiment, the sample image set stores both labeled and unlabeled sample images. Both types of sample images carry realistic and effective scene styles and visual features, and both participate in semantic feature similarity calculation. There are no priority or filtering restrictions for sample types. Therefore, the first K images selected based on similarity from high to low may all be labeled sample images with region annotation information, i.e., the first candidate sample images of the first category; or they may all be unlabeled sample images without annotation information, i.e., the second candidate sample images; or they may contain both the first candidate sample images of the first category and the second candidate sample images. The above selection logic, without manually filtering sample types, selects samples uniformly based on feature similarity, which can retain realistic style samples that highly match the features of the input image to the greatest extent, avoiding the omission of high-match style reference samples due to manually setting sample type selection conditions. This is beneficial to ensuring the integrity and richness of the style reference dimension on which style enhancement processing depends.

[0097] Step S1022: Determine that the second candidate sample image exists, and obtain a second type of first candidate sample image that matches the second candidate sample image from the sample image set; the second type of first candidate sample image has annotation information.

[0098] If the existence of the second candidate sample image is determined, the same feature similarity calculation method described above can be used to retrieve a sample image in the sample image set that matches the features of the second candidate sample image and has annotation information, and the sample image is regarded as the second type of first candidate sample image.

[0099] It should be noted that the number of samples K selected in this step of the retrieval and matching process can be flexibly adjusted according to the actual task requirements and the size of the sample set, and does not need to be consistent with the number selected in step S1021.

[0100] In this embodiment, by sorting and selecting the first candidate sample image and / or the second candidate sample image that match the input image based on the similarity between the input image and each real image in the sample image set in the feature space, the first candidate sample image and / or the second candidate sample image that match the input image can be selected uniformly based on feature similarity without manually filtering the sample types. This maximizes the retention of real style samples that are highly consistent with the features of the input image, effectively avoiding the omission of high-matching style reference samples due to manually setting sample type filtering conditions, and optimizing the comprehensiveness and adaptation accuracy of the first candidate sample image selection.

[0101] Based on this, after confirming the existence of the second candidate sample image, which is a real-world scene sample that highly matches the semantic features of the input image, its overall scene distribution and visual style are highly compatible with the input image, possessing extremely high style reference value. However, limited by its lack of region annotation, it cannot autonomously complete semantic region segmentation and region style quantification, and cannot directly participate in the region-specific style enhancement operation. Therefore, a second type of first candidate sample image with features matching the second candidate sample image and possessing region annotation information is further retrieved from the sample image set. Since the second type of first candidate sample image has a highly similar semantic style to the second candidate sample image, the style of each region of the second candidate sample image can be accurately determined based on the second type of first candidate sample image. This transforms the unlabeled sample style resources, which were originally unusable but highly compatible with the input image, into effective region-specific style references. This method can further expand the number and style diversity of the first candidate sample images without increasing the region annotation cost.

[0102] In this embodiment, combined with Figure 3 The process of determining the first candidate sample image of the first category and the second candidate sample image of the second category is explained. For example, the sample image set includes sample images with annotation information (which can be represented as x1). L x2 L x3 L ... x nL L) and sample images without annotation information (which can be represented as x1) U x2 U x3 U ... x nU U First, determine the relationship between each sample image in the sample image set and the input image x. i S The similarity between the images is calculated, and the images are sorted from highest to lowest similarity. The top K images are selected. For example, with K=3, the top three selected images include the first candidate image of the first category (i.e., x1). L x3 L ) and the second candidate sample image (i.e., x1) U (The first candidate sample image of the first category has annotation information, while the second candidate sample image does not have annotation information.)

[0103] Due to the second candidate sample image x1 U Lacking annotation information, it cannot autonomously complete semantic region segmentation and region style quantization, requiring further acquisition of a second type of first candidate sample image matching it from the sample image set. Therefore, the same similarity calculation method as described above is used, and the annotated sample images (i.e., x1) in the sample image set are used. L x2 L x3 L ... x nL L In the calculation, the labeled sample image and the second candidate sample image x1 are compared. U The similarity between the images is calculated, and the images are sorted from highest to lowest similarity. The top K labeled sample images are selected. For example, if K=1, the first labeled sample image selected would be x2. L x2 L This is the second category of first candidate sample images. The second category of first candidate sample images has annotation information and is related to the second candidate sample image x1. U The semantics are highly similar, and it can be based on the second type of first candidate sample image x2. L The annotation information is used to accurately determine the second candidate sample image x1. U The style parameters of each region are used to transform unlabeled sample style resources, which were originally unusable but highly adapted to the input image, into effective regional style references.

[0104] It should be noted that the storage methods for sample images with and without annotation information in the sample image set are not limited to... Figure 3 The two can be displayed separately or stored together in the same set of sample images. Figure 3The way labeled and unlabeled sample images are listed separately is only for illustrating the differences in attributes of each sample image and does not constitute a restriction on the storage method of the sample image set. Regardless of the storage method used, when executing step S1021, all sample images in the sample image set participate in the similarity calculation and sorting selection with the input image, and are not subject to screening priority or filtering restrictions due to differences in storage format or labeled attributes.

[0105] As another optional embodiment of this application, refer to Figure 4 This is a flowchart illustrating an image generation method provided in Embodiment 3 of this application. This embodiment is mainly an implementation of step S103 in Embodiment 2, such as... Figure 4 As shown, it may include, but is not limited to:

[0106] Step S1031: Based on the annotation information of the first candidate sample image of the first type and the first candidate sample image of the second type respectively, determine the style parameters corresponding to at least two regions of the first candidate sample image of the first type and the first candidate sample image of the second type respectively.

[0107] In this embodiment, the first candidate sample image of the first category can be accurately divided into at least two regions based on the value of each pixel in the annotation information of the first candidate sample image of the first category. The pixel ownership relationship and region boundary of each region are uniquely determined by the annotation information. The partitioning result is unambiguous and closely fits the inherent semantic structure of the image.

[0108] For example, the annotation information of the first candidate sample image of the first category may include a first segmentation mask M1, where the value of each pixel in the first segmentation mask M1 is used to indicate the semantic region category to which the corresponding pixel belongs. The first candidate sample image of the first category can be segmented according to the first segmentation mask M1. Separate into the first foreground region:

[0109]

[0110] First background area:

[0111]

[0112] in, This represents the first foreground region of the first candidate sample image of the first class. This represents the first background region of the first candidate sample image of the first category, and ⊙ represents pixel-by-pixel multiplication. First foreground region. The pixel values ​​of each pixel in the first candidate sample image of the first class are compared with those of the first pixel in the first candidate sample image. The pixel values ​​at corresponding positions are consistent, in the first background area. The pixel values ​​of each pixel in the first candidate sample image of the first class are compared with those of the first pixel in the first candidate sample image. The pixel values ​​at corresponding positions are consistent, and the pixel affiliation between the two is uniquely determined by the first segmentation mask M1.

[0113] Similarly, based on the pixel values ​​in the annotation information of the second type of first candidate sample image, the second type of first candidate sample image can be accurately divided into at least two regions. The pixel ownership relationship and region boundary of each region are uniquely determined by the annotation information, and the partitioning result is unambiguous and closely fits the inherent semantic structure of the image.

[0114] For example, the annotation information of the second type of first candidate sample image may include a second segmentation mask M2, which can be used to segment the second type of first candidate sample image. Separate into a second foreground region and a second background region:

[0115]

[0116]

[0117] in, This represents the second foreground region of the first candidate sample image of the second category. This represents the second background region of the first candidate sample image in the second category. Second foreground region. With the second background area The pixel affiliation is uniquely determined by the second segmentation mask M2, and the partitioning result closely matches the inherent semantic structure of the image.

[0118] In this embodiment, the style parameters of a region can be used to quantitatively characterize the style of the region.

[0119] The style parameters for determining a region may include, but are not limited to, at least one of the following:

[0120] Determine the mean of the region. The mean can characterize the central tendency of the region's style.

[0121] Determine the standard deviation of the region. The standard deviation can characterize the dispersion of regional styles.

[0122] Step S1032: Based on the style parameters corresponding to at least two regions of the second type of first candidate sample image and the similarity between the second type of first candidate sample image and the second candidate sample image, determine the style parameters corresponding to at least two regions of the second candidate sample image.

[0123] The second category of first candidate sample images are merely reference samples semantically similar to the second candidate sample images, not perfectly matching scene images. They exhibit inherent subtle differences in illumination intensity, texture detail, color distribution, and imaging state. Directly reusing the fixed region style parameters of the second category of first candidate sample images would ignore the unique scene style features of the second candidate sample images themselves, resulting in stiff style matching, insufficient adaptability, and introducing additional style shift errors, failing to accurately match the true style attributes of the second candidate sample images. However, by determining the style parameters corresponding to at least two regions of the second candidate sample image based on their similarity, it is possible to accurately adapt to the unique style features of the second candidate sample image, achieving high-precision, adaptive determination of region style parameters.

[0124] In this embodiment, the similarity calculation formula in step S1021 of embodiment 2 can be used to determine the similarity between the second type of first candidate sample image and the second candidate sample image, which will not be repeated here.

[0125] Step S1033: Based on the style parameters corresponding to at least two regions of the first candidate sample image and the second candidate sample image, and the similarity between the first candidate sample image and the second candidate sample image and the input image, determine the style parameters corresponding to at least two regions of the input image.

[0126] In this embodiment, the regional style parameters of the first candidate sample image of the first category are directly extracted from its own annotation information. It belongs to the original and accurate standard style benchmark and has high credibility. However, due to the limited total number of labeled samples, the style variations it can cover are relatively limited. Relying solely on labeled sample images will lead to the style template becoming fixed and difficult to adapt to the complex and ever-changing style requirements of real scenes.

[0127] The region style parameters of the second candidate sample image (unlabeled sample image) are determined based on the region style parameters of the second type of first candidate sample image (labeled sample image) that are similar to it. It carries rich real scene style details, which can effectively make up for the lack of single style dimension of labeled samples. However, since it is not directly extracted, it cannot be used as a benchmark for style correction on its own.

[0128] Based on the similarity between the first candidate sample image and the second candidate sample image of the first class and the input image, the region style parameters of the two classes of samples can be fused to obtain region style parameters that match the true matching degree of the input image.

[0129] Step S1034: Based on the style parameters corresponding to at least two regions of the input image, perform enhancement processing on at least two regions of the input image to obtain at least two enhanced regions.

[0130] In this embodiment, the input image may have annotation information, which is used to distinguish at least two regions of the input image.

[0131] The annotation information of the input image can be obtained in the following way: when the input image is generated as a synthetic image, a corresponding segmentation mask can be generated simultaneously. The segmentation mask exists in the form of a mask image of the same size as the input image. The value of each pixel in the mask image corresponds to the semantic region category to which it belongs in the input image, thereby achieving pixel-level accurate division of each region of the input image.

[0132] Alternatively, a pre-trained semantic segmentation model can be used to automatically segment the input image into regions and generate corresponding annotation information.

[0133] For example, the annotation information of the input image may include a segmentation mask M, and the input image is segmented according to the segmentation mask M. Separate into foreground and background regions:

[0134]

[0135]

[0136] in, This represents the foreground region of the input image. The symbol represents the background region of the input image, and circle (⊙) represents pixel-by-pixel multiplication. The foreground region... The pixel values ​​of each pixel in the input image The pixel values ​​at corresponding positions are consistent, and the background area... The pixel values ​​of each pixel in the input image The pixel values ​​at corresponding positions are consistent, and the pixel affiliation between the two is uniquely determined by the segmentation mask M. The partitioning result is unambiguous and closely fits the inherent semantic structure of the input image.

[0137] In this embodiment, the input image region is affected by the synthetic imaging mechanism and generally suffers from inherent style shift defects. The color distribution, brightness performance, and texture details of each semantic region deviate from the imaging characteristics of the real scene, resulting in problems such as color temperature distortion, uneven brightness, false texture, and mixed styles. Furthermore, the differentiated style features of different semantic regions are covered by a unified synthetic style, which fails to reflect the inherent rules of independent distribution of regional styles in the real scene.

[0138] After enhancing at least two regions of the input image based on the style parameters corresponding to each of the at least two regions of the input image, each independent region of the input image is adapted to its corresponding real-world scene style.

[0139] In this embodiment, by determining the style parameters corresponding to at least two regions of each candidate sample image based on the annotation information of the first type of first candidate sample image and the second type of first candidate sample image respectively, the annotation information can be used to perform accurate and unambiguous region division of each candidate sample image. The partitioning result closely fits the inherent semantic structure of the image and does not require repeated iterative calculation or adjustment of threshold parameters, which significantly reduces the uncertainty of style parameters and further improves the overall efficiency and stability of region division and style parameter extraction.

[0140] Based on this, the style parameters of each region of the second candidate sample image are determined according to the similarity between the second type first candidate sample image and the unlabeled second candidate sample image. This can adapt the accurate regional style parameters of the second type first candidate sample image, which is highly similar to the second candidate sample image in semantics, to the unlabeled sample. This avoids the insufficient adaptability and additional style shift error caused by directly reusing fixed style parameters and ignoring the subtle style differences between samples. It achieves high-precision and adaptive quantization of the unique style features of unlabeled samples.

[0141] Furthermore, by integrating the native precise style parameters of the first candidate sample image of the first category with the style parameters determined by adaptation of the second candidate sample image, and combining their similarity with the input image to determine the style parameters of each region of the input image, it can comprehensively combine the standard benchmark of labeled samples with the rich real style details of unlabeled samples. This effectively makes up for the problem of style template solidification caused by relying solely on labeled samples, so that the style parameters of each independent region of the input image can accurately adapt to the inherent distribution law of its real scene. While eliminating the native invalid visual interference and false style information of the synthetic image, it significantly improves the defects of style distortion and poor adaptability of the input image, and achieves higher precision regional style enhancement processing.

[0142] As another optional embodiment of this application, refer to Figure 5 This is a flowchart illustrating an image generation method provided in Embodiment 4 of this application. In this embodiment, there are multiple first candidate sample images of the second type. This embodiment is mainly an implementation of step S1032 in Embodiment 3, such as... Figure 5 As shown, it may include, but is not limited to:

[0143] Step S11: Based on the similarity between each second-class first candidate sample image and the second candidate sample image, determine the first weight factor corresponding to each second-class first candidate sample image; the first weight factor characterizes the relative similarity between the second candidate sample image and the second-class first candidate sample image.

[0144] When obtaining a second type of first candidate sample image that matches the second candidate sample image from the sample image set, multiple second type of first candidate sample images with the highest similarity ranking to the second candidate sample image can be selected.

[0145] Since the similarity between each second-category first-candidate image and the second-candidate image varies, the second-category first-candidate image with higher similarity to the second-candidate image has greater reference value for its region style parameters and should occupy a higher weight ratio in the fusion process. Therefore, this embodiment can use the softmax function to normalize the similarity between each second-category first-candidate image and the second-candidate image, converting the similarity into a first weight factor that can be directly used for weighted fusion.

[0146] In this embodiment, the first weighting factor can be determined using the following relationship:

[0147]

[0148] in, This represents the m-th candidate sample image of the second class. Compared to the second candidate sample image The first weighting factor, Indicates the second candidate sample image With the m-th second-class first candidate sample image Similarity in feature space This is a feature extraction function used to extract semantic features from an image. Indicates from the set of sample images Image of the middle and second candidate samples The candidate subset consists of the top K similarity-ranked second-class first-candidate sample images, where n is the number of candidate subsets traversed. The index variables of each second-class first-candidate sample image in the middle. This represents the nth candidate image of the second class in the candidate subset.

[0149] Softmax normalization makes the sum of all first weight factors equal to 1, and the first weight factor corresponding to the second category first candidate sample image with higher similarity is larger, while the first weight factor corresponding to the image with lower similarity is smaller, thus realizing adaptive weight allocation based on similarity.

[0150] Step S12: Based on the first weighting factor, the style parameters corresponding to at least two regions of each of the second type of first candidate sample images are fused to obtain the style parameters corresponding to at least two regions of the second candidate sample image.

[0151] In this embodiment, the style parameters corresponding to at least two regions of each second-class first candidate sample image may include, but are not limited to, the mean μ and standard deviation σ of the region.

[0152] In this embodiment, the style parameters of each second-class first candidate sample image in the same region can be weighted and averaged according to the first weighting factor to obtain the style parameters of the second candidate sample image in that region.

[0153] For example, the mean of the second candidate sample image in the foreground or background region can be determined using the following relationship. and standard deviation :

[0154]

[0155]

[0156] in, and These are the m-th second-class first-candidate sample images. The mean and standard deviation over region r, where r represents the foreground or background region. The first weight factor is the first weight factor corresponding to the m-th second-class first-candidate sample image. Indicates from the set of sample images Image of the middle and second candidate samples The candidate subset consists of the K most similar first-class candidate sample images, where m is the number of candidate subsets traversed. The index variables of the first candidate sample images of each second category.

[0157] In this embodiment, by using a weighted averaging method, the region style parameters of the second-class first-candidate sample image with higher similarity to the second-candidate sample image contribute more to the fusion result. This allows the style parameters of each region in the final determined second-candidate sample image to more accurately match its true style attributes. Compared to directly selecting the style parameters of a single second-class first-candidate sample image or simply averaging the style parameters of multiple second-class first-candidate sample images, this step, through a similarity-driven weighted fusion mechanism, effectively reduces the style shift error introduced by significant style differences between individual second-class first-candidate sample images and the second-candidate sample image, thus improving the accuracy and robustness of determining the style parameters of unlabeled sample regions.

[0158] As another optional embodiment of this application, refer to Figure 6 This is a flowchart illustrating an image generation method provided in Embodiment 5 of this application. This embodiment is mainly an implementation of step S1033 in Embodiment 3, such as... Figure 6 As shown, it may include, but is not limited to:

[0159] Step S21: Based on the similarity between the first candidate sample image and the second candidate sample image of the first class and the input image respectively, determine the second weight factor and the third weight factor corresponding to the first candidate sample image and the second candidate sample image of the first class respectively.

[0160] The second weighting factor characterizes the relative similarity between the first candidate sample image of the first class and the input image.

[0161] The third weighting factor characterizes the relative similarity between the second candidate sample image and the input image.

[0162] Since the similarity between the first and second candidate sample images and the input image varies, the sample image with higher similarity to the input image has greater reference value for its region style parameters and should therefore occupy a higher weight ratio in the fusion process. Therefore, in this embodiment, the softmax function can be used to normalize the similarity between the first and second candidate sample images and the input image, converting the similarity into second and third weighting factors that can be directly used for weighted fusion.

[0163] In this embodiment, the second and third weighting factors can be determined in the same way as the first weighting factor in Embodiment 4, and will not be repeated here.

[0164] Step S22: Based on the second weighting factor and the third weighting factor, the style parameters corresponding to at least two regions of the first candidate sample image and the second candidate sample image are fused to obtain the style parameters corresponding to at least two regions of the input image.

[0165] In this embodiment, for each of the at least two regions, the style parameters of the first candidate sample image and the second candidate sample image in the same region can be weighted and averaged based on the second weighting factor and the third weighting factor to obtain the style parameters of the input image in that region.

[0166] For example, the mean of the input image in the foreground or background region can be determined using the following relationship. and standard deviation :

[0167]

[0168]

[0169] in, and Here, represents the mean and standard deviation of the first candidate sample image of the first class in region r, respectively. and , respectively, represent the mean and standard deviation of the second candidate sample image in region r, where r represents the foreground or background region. As the second weighting factor, It is the third weighting factor.

[0170] In this embodiment, by determining the corresponding second and third weighting factors based on the similarity between the first candidate sample image and the second candidate sample image and the input image, the similarity can be transformed into a weight allocation basis that can be directly used for weighted fusion. This allows sample images with higher similarity to the input image to occupy a larger weight proportion during the fusion process. Furthermore, by weighted averaging the style parameters of the first candidate sample image and the second candidate sample image in the same region, the style of labeled samples and the rich style details determined through adaptation of unlabeled samples can be integrated. This effectively compensates for the style template solidification problem caused by relying solely on labeled samples and avoids style shift errors introduced by significant differences between individual reference samples and the input image style. This further improves the accuracy, robustness, and adaptability to complex and varied real-world scene styles in determining the style parameters of each region of the input image.

[0171] As another optional embodiment of this application, refer to Figure 7 This is a flowchart illustrating an image generation method provided in Embodiment 6 of this application. This embodiment is mainly an implementation of step S1034 in Embodiment 3, such as... Figure 7 As shown, it may include, but is not limited to:

[0172] Step S31: Based on the style parameters corresponding to at least two regions of the input image, perform style normalization processing on at least two regions of the input image.

[0173] In this embodiment, style normalization can be performed on each of at least two regions of the input image based on the style parameters corresponding to that region. For example, for the foreground region segmented by the segmentation mask in the input image, normalization is performed based on the style parameters corresponding to the foreground region; for the background region, normalization is performed based on the style parameters corresponding to the background region. The style normalization processes for each region can be independent of each other, each using its own style parameters as the calculation basis.

[0174] Style normalization of regions can weaken the inherent style shifts of each region in the input image that deviate from the real scene, remove invalid original style information of the image, and retain the effective information corresponding to the image content and structure, thus obtaining the normalized regions.

[0175] Step S32: Based on the style parameters corresponding to at least two regions of the input image, perform style reconstruction processing on the at least two regions after normalization to obtain at least two regions after enhancement processing.

[0176] In this embodiment, for each normalized region of the input image, style mapping can be performed on the normalized region using the style parameters corresponding to that region, so that the true style attributes of the region are assigned to the normalized region.

[0177] For example, for the foreground region of an input image, style normalization and style reconstruction can be performed using the following relationship:

[0178]

[0179] For the foreground region of the input image, the style parameter, i.e., the mean, corresponding to the foreground region of the input image determined in the aforementioned step S1033 is... and standard deviation As a normalization parameter, style normalization is first performed, that is, through... , foreground area The pixel value distribution is centered and standardized so that the mean of the normalized foreground region is 0 and the standard deviation is 1, thereby eliminating the deviation of the original composite style of the foreground region in terms of absolute mean and variance.

[0180] Based on this, the style parameter corresponding to the foreground region of the input image determined in step S1033 is the mean. and standard deviation As a parameter for refactoring, style refactoring is performed, that is, through... The normalized pixel values ​​of the foreground region are reconstructed to enhance the foreground region. The mean was adjusted to The standard deviation was adjusted to In other words, the style of the foreground area accurately matches the style that the foreground area should have in a real scene.

[0181] For the background region of the input image, style normalization and style reconstruction can be performed using the following relationship:

[0182]

[0183] For the background region of the input image, the style parameter, i.e., the mean, corresponding to the background region of the input image determined by the aforementioned step S1033 is... and standard deviation As a normalization parameter, style normalization is first performed, that is, through... Background area The pixel value distribution is centered and standardized so that the mean of the normalized background area is 0 and the standard deviation is 1, thereby eliminating the deviation of the original composite style of the background area in terms of absolute mean and variance.

[0184] Based on this, the style parameter corresponding to the background region of the input image determined in step S1033 is the mean. and standard deviation As a parameter for refactoring, style refactoring is performed, that is, through... The normalized background pixel values ​​are reconstructed to enhance the background region. The mean was adjusted to The standard deviation was adjusted to In other words, the style of the background area accurately matches the style that the background area should have in a real scene.

[0185] In this embodiment, by performing style normalization processing on at least two regions of the input image respectively, the inherent style shifts of each region of the input image that deviate from the real scene can be weakened, and the original invalid style information of the image can be removed, retaining only the valid information corresponding to the content and structure. On this basis, the same set of style parameters is further used to perform style reconstruction processing on the normalized regions, which can accurately map the style attributes of the real scene to the content structure that has removed the original style interference. Since both the normalization and reconstruction stages use the same set of style parameters determined by the aforementioned steps and accurately adapted to the real matching degree of the input image, the style feature drift or secondary distortion that may be caused by staged processing is avoided. This effectively overcomes the defects of cross-regional style interference and semantic distortion caused by the global correction method ignoring the independence of regional styles, and achieves high-precision, decoupled style enhancement for each independent region of the input image, significantly improving the realism and naturalness of the generated image in terms of style representation.

[0186] As another optional embodiment of this application, refer to Figure 8 This is a flowchart illustrating an image generation method provided in Embodiment 7 of this application. This embodiment is mainly an implementation of step S104 in Embodiment 1, such as... Figure 8 As shown, it may include, but is not limited to:

[0187] Step S1041: Obtain the annotation information of the input image; the annotation information of the input image includes at least two regions used to distinguish the input image.

[0188] For a detailed description of step S1041, please refer to the relevant introduction on the annotation information of the input image in step S1034 of Embodiment 3, which will not be repeated here.

[0189] Step S1042: Based on the annotation information of the input image, perform noise reduction processing on at least two regions after enhancement processing.

[0190] While the style of the enhanced regions in the input image has been adapted to the corresponding real-world scene style, invalid visual noise or artifacts may still be introduced at region boundaries and within each region. For example, color abruptness or brightness discontinuity may occur at region boundaries due to differences in style parameters. If a global denoising method without annotation is used, it will be impossible to distinguish semantic region boundaries. While eliminating noise, it will inevitably erase the effective style details that have been accurately adapted after style reconstruction in each region, resulting in a weakened style enhancement effect and damage to the semantic structure.

[0191] By performing region-based denoising based on the annotation information of the input image, the denoising range can be precisely defined by semantic region. It only suppresses invalid noise introduced within each region, while fully preserving the effective style details and semantic features of each region after style reconstruction.

[0192] For example, the enhanced foreground region can be denoised using the following relationship:

[0193]

[0194] This indicates that the foreground region is weighted by the segmentation mask M of the foreground region after enhancement. Perform pixel-level filtering: At the pixel position corresponding to the foreground area, the mask M is set to 1, and the calculation result is retained. The original pixel value at this location, i.e., the effective style details of the foreground region, are fully preserved, enabling denoising of the foreground region and resulting in the denoised foreground region. .

[0195] The enhanced background region can be denoised using the following relationship:

[0196]

[0197] This indicates that the background region is enhanced by using the segmentation mask (1-M) as the weight. Perform pixel-level filtering: At the pixel position corresponding to the background region, the background segmentation mask (1-M) is set to 1, and the calculation result is retained. The original pixel values ​​at this location, i.e., the effective style details of the background area, are fully preserved, thus achieving noise reduction of the background area and obtaining the noise-reduced background area. .

[0198] Step S1043: Fuse at least two regions after denoising to obtain the target image.

[0199] In this embodiment, the pixel values ​​of each region after noise reduction can be added together at their original spatial positions to obtain the target image.

[0200] For example, the denoised foreground and background regions can be merged using the following relationship:

[0201]

[0202] This indicates that the effective pixels in the foreground region and the effective pixels in the background region have been added pixel-level at their respective spatial locations to form a complete target image.

[0203] In this embodiment, based on the annotation information of the input image, at least two regions after enhancement are subjected to regional denoising. Region fusion is achieved through mask-guided pixel-level filtering and addition operations. This allows for precise definition of the denoising range at the semantic region level. While effectively suppressing invalid visual noise and artifacts introduced by style transfer operations at region boundaries and within each region, it fully preserves the effective style details and semantic features that have been accurately adapted after style reconstruction in each region. This avoids the problem of effective style detail erasure and semantic structure damage caused by the inability to distinguish semantic region boundaries in global denoising under unannotated conditions. Furthermore, the effective pixels of each denoised region are precisely added at the pixel level in their original spatial positions. The fused target image has neither pixel omissions nor pixel overlaps. The boundaries between the foreground and background regions are naturally continuous without any stitching marks. The final generated target image is completely consistent with the input image in semantic content and closely approximates the distribution of the real scene in style representation, significantly improving the visual naturalness and structural fidelity of the style shift correction results.

[0204] In this embodiment, combined with Figure 9 The image generation method is explained. For example, such as... Figure 9As shown, the input image is an insulator image, which is a composite image and has a segmentation mask. The segmentation mask is one implementation of annotation information. Based on the segmentation mask, the input image can be accurately divided into a foreground region and a background region, where the foreground region corresponds to the insulator body and the background region corresponds to the environment background such as the sky.

[0205] The insulator image is a composite image. If it is necessary to generate an insulator image with a visual style under sunlight illumination, a first candidate sample image can be selected from the sample image set under sunlight illumination (which may include sample images with annotation information and sample images without annotation information). Based on the annotation information of the first candidate sample image, the first candidate sample image is divided into a foreground region and a background region. Then, the mean m1 and standard deviation a1 of the foreground region of the first candidate sample image and the mean m2 and standard deviation a2 of the background region of the first candidate sample image are determined.

[0206] The foreground region of the input image is style-enhanced based on the mean m1 and standard deviation a1 of the foreground region of the first candidate sample image, resulting in the enhanced foreground region (which can be represented as Image1).

[0207] The background region of the input image is style-enhanced based on the mean m2 and standard deviation a2 of the background region of the first candidate sample image, resulting in the enhanced background region (which can be represented as Image2).

[0208] The segmentation mask based on the input image can be used to denoise the enhanced foreground and background regions separately. The denoised foreground and background regions are then merged to obtain the target image. The style distribution of the foreground region of this target image matches the realistic style characteristics of an insulator under sunlight illumination, while the style distribution of the background region matches the realistic style characteristics of environmental backgrounds such as the sky under sunlight illumination. This results in the entire target image visually exhibiting style attributes consistent with a real scene under sunlight illumination.

[0209] This embodiment, through independent enhancement of different regions and mask-guided denoising fusion, can accurately eliminate artifacts such as color jumps and brightness discontinuities at the boundaries of regions caused by differences in style parameters, while preserving the effective style details of the insulator body and the background. This ensures that the target image is completely consistent with the input image in terms of semantic content and fully reflects the distribution of the real scene under sunlight illumination in terms of style, significantly improving the visual naturalness and structural fidelity of the synthesized image in terms of style correction to the real scene.

[0210] As another optional embodiment of this application, an image generation method is provided in Embodiment 8 of this application. This embodiment is mainly an implementation of the method in Embodiment 2 for obtaining a second type of first candidate sample image that matches the second candidate sample image from the sample image set. Specifically, it may include, but is not limited to:

[0211] Step S41: Use the second candidate sample image as the current query image.

[0212] Step S42: Query at least one third candidate sample image from the sample image set that is similar to the currently queried image.

[0213] In this embodiment, the query process can refer to the similarity-based selection process in the previous embodiments, and will not be repeated here.

[0214] Step S43: If none of the at least one third candidate sample image has annotation information, update the current query image based on the at least one third candidate sample image, and continue to execute the step of selecting at least one third candidate sample image similar to the current query image from the sample image set, until the at least one third candidate sample image contains a second type of first candidate sample image or the set number of queries is reached.

[0215] In this embodiment, when the second candidate sample image initially selected from the sample image set does not have annotation information, it is not possible to directly perform regional style enhancement processing based on the second candidate sample image. Therefore, it is necessary to gradually approximate the second type of first candidate sample image with style matching and annotation information in the sample image set through iterative query.

[0216] If at least one of the third candidate sample images contains a sample image with annotation information, then the sample image with annotation information is determined as the second type of first candidate sample image, and the query process ends. If none of the at least one third candidate sample image contains annotation information, then the current query image is updated based on at least one third candidate sample image. That is, the sample image most similar to the original current query image among the at least one third candidate sample image is taken as the new current query image, and at least one third candidate sample image similar to the updated current query image is queried again from the sample image set. This process is repeated until the queried third candidate sample images contain a second type of first candidate sample image with annotation information, or the number of iterations reaches the preset query limit.

[0217] For example, select three sample images similar to the input image from the sample image set, namely sample image A, sample image B and sample image C. Sample images A and B have annotation information and can be regarded as first-class candidate sample images, while sample image C does not have annotation information and can be regarded as second-class candidate sample images.

[0218] Since sample image C lacks annotation information and cannot be directly used as a source of style parameters, sample image C is used as the current query image. The system continues to search for sample images similar to sample image C in the sample image set. For example, if sample image D is found, sample image D can be considered as the third candidate sample image. If sample image D has annotation information, then sample image D is determined as the second type of first candidate sample image that matches the second candidate sample image C, and the query process ends.

[0219] If sample image D does not have annotation information, then update the current query image with sample image D, and continue to query sample images similar to sample image D in the sample image set. Repeat the above process until a sample image with annotation information is found or the set number of queries is reached.

[0220] Through the above iterative query mechanism, even if there is no directly matching sample image with labeled information in the vicinity of the sample image set, the second candidate sample image can be located by gradually jumping to sample images with similar styles. This ensures that the subsequent regional style enhancement processing has a reliable source of style parameters and avoids the style enhancement process from failing to execute due to the lack of labeled information in the second candidate sample image.

[0221] As another optional embodiment of this application, an image generation method provided in Embodiment 9 of this application is mainly an implementation of updating the current query image based on the at least one third candidate sample image in Embodiment 8. Specifically, it may include, but is not limited to, any one of the following:

[0222] Step S431: Select the third candidate image with the highest similarity to the current query image from the at least one third candidate image as the current query image.

[0223] In this embodiment, the similarity between each third candidate sample image in at least one third candidate sample image and the original current query image can be determined, and the third candidate sample image with the highest similarity can be selected and directly used as the new current query image for the next round of query.

[0224] Among multiple third candidate sample images that do not have annotation information, the sample image with the highest similarity to the current query image has the style features that are closest to the original current query image. Using it as the new query starting point, it can continue to approach the sample image with annotation information with the smallest style shift, thereby locating the second type of first candidate sample image with annotation information and style matching with the fewest iterations, thus improving query efficiency.

[0225] Step S432: Based on the similarity between each third candidate sample image in the at least one third candidate sample image and the current query image, fuse each third candidate sample image, and use the fused image as the current query image.

[0226] In this embodiment, the similarity between each third candidate sample image and the current query image can be used as the fusion weight to perform weighted fusion on at least one third candidate sample image to generate a fused image, and the fused image can be used as the new current query image for the next round of query.

[0227] A single maximum similarity sample image may cause the query direction to deviate due to individual biases. However, by using similarity-weighted fusion, the style features of multiple third candidate sample images can be integrated, making the new current query image more robust and representative in style. This reduces the risk of deviation in the iteration direction due to style anomalies of a single sample image in subsequent queries, and improves the reliability of finally locating the second type of first candidate sample image with labeled information and style matching.

[0228] In practical applications, either step S431 or step S432 can be executed, or they can be used alternately in different iterations. When high query efficiency is required, the maximum similarity selection strategy of step S431 can be used first to approximate the target sample image as quickly as possible. When high style accuracy of the query results is required, the similarity weighted fusion strategy of step S432 can be used first to approximate the target sample image in a more robust way, ensuring that the final obtained second-class first candidate sample image is highly consistent with the original second candidate sample image in style, providing a reliable source of style parameters for subsequent regional style enhancement processing.

[0229] The image generation apparatus provided in this application will be described below. The image generation apparatus described below can be referred to in correspondence with the image generation method described above.

[0230] The image generation device includes: a first acquisition module, a second acquisition module, an enhancement module, and a generation module.

[0231] The first acquisition module is used to acquire the input image.

[0232] The second acquisition module is used to acquire at least one first candidate sample image that matches the input image from the sample image set; the at least one first candidate sample image has annotation information, which is used to distinguish at least two regions of the first candidate sample image; wherein, the first candidate sample image contains part or all of the sample images in the sample image set that match the second candidate sample image; the second candidate sample image matches the input image but does not have annotation information.

[0233] An enhancement module is used to perform style enhancement processing on at least two regions of the input image based on the at least one first candidate sample image, to obtain at least two enhanced regions.

[0234] The generation module is used to generate a target image based on at least two regions after the enhancement process.

[0235] In this embodiment, the second acquisition module can specifically be used for:

[0236] Obtain a first-class first-candidate sample image and / or a second-candidate sample image that matches the input image from a set of sample images; the first-class first-candidate sample image has annotation information; the second-candidate sample image does not have annotation information; the annotation information is used to distinguish at least two regions of the corresponding candidate sample image;

[0237] Once the existence of the second candidate sample image is determined, a second type of first candidate sample image matching the second candidate sample image is obtained from the sample image set; the second type of first candidate sample image has annotation information.

[0238] In this embodiment, the enhancement module can specifically be used for:

[0239] Based on the annotation information of the first candidate sample image of the first type and the first candidate sample image of the second type respectively, determine the style parameters corresponding to at least two regions of the first candidate sample image of the first type and the first candidate sample image of the second type respectively;

[0240] Based on the style parameters corresponding to at least two regions of the second type of first candidate sample image, and the similarity between the second type of first candidate sample image and the second candidate sample image, the style parameters corresponding to at least two regions of the second candidate sample image are determined.

[0241] Based on the style parameters corresponding to at least two regions of each of the first candidate sample image and the second candidate sample image, and the similarity between each of the first candidate sample image and the second candidate sample image and the input image, the style parameters corresponding to at least two regions of the input image are determined.

[0242] Based on the style parameters corresponding to at least two regions of the input image, enhancement processing is performed on at least two regions of the input image to obtain at least two enhanced regions.

[0243] In this embodiment, there can be multiple first candidate sample images of the second type; the enhancement module determines the style parameters corresponding to at least two regions of the second candidate sample image based on the style parameters corresponding to at least two regions of each of the second candidate sample images of the second type, and the similarity between the second candidate sample image of the second type and the second candidate sample image. Specifically, this may include:

[0244] Based on the similarity between each second-class first candidate sample image and the second candidate sample image, a first weight factor is determined for each second-class first candidate sample image; the first weight factor characterizes the relative similarity between the second candidate sample image and the second-class first candidate sample image.

[0245] Based on the first weighting factor, the style parameters corresponding to at least two regions of each of the second type of first candidate sample images are fused to obtain the style parameters corresponding to at least two regions of the second candidate sample image.

[0246] The enhancement module determines the style parameters corresponding to at least two regions of the input image based on the style parameters corresponding to at least two regions of each of the first candidate sample image and the second candidate sample image, and the similarity between each of the first candidate sample image and the second candidate sample image and the input image. Specifically, this can include:

[0247] Based on the similarity between the first candidate sample image and the second candidate sample image and the input image, a second weighting factor and a third weighting factor are determined for the first candidate sample image and the second candidate sample image, respectively; the second weighting factor represents the relative similarity between the first candidate sample image and the input image; the third weighting factor represents the relative similarity between the second candidate sample image and the input image.

[0248] Based on the second weighting factor and the third weighting factor, the style parameters corresponding to at least two regions of the first candidate sample image and the second candidate sample image are fused to obtain the style parameters corresponding to at least two regions of the input image.

[0249] The enhancement module performs enhancement processing on at least two regions of the input image based on style parameters corresponding to each of the at least two regions, resulting in at least two enhanced regions. Specifically, this may include:

[0250] Based on the style parameters corresponding to at least two regions of the input image, style normalization processing is performed on at least two regions of the input image.

[0251] Based on the style parameters corresponding to at least two regions of the input image, style reconstruction processing is performed on the at least two regions after normalization to obtain at least two regions after enhancement processing.

[0252] In this embodiment, the generation module can specifically be used for:

[0253] Obtain the annotation information of the input image; the annotation information of the input image includes at least two regions used to distinguish the input image;

[0254] Based on the annotation information of the input image, denoising processing is performed on at least two regions after the enhancement process;

[0255] At least two regions after denoising are fused to obtain the target image.

[0256] In this embodiment, the second acquisition module acquires a second type of first candidate sample image that matches the second candidate sample image from the sample image set, specifically including:

[0257] Use the second candidate sample image as the current query image;

[0258] Query at least one third candidate sample image from the set of sample images that is similar to the currently queried image;

[0259] If none of the at least one third candidate sample image has annotation information, the current query image is updated based on the at least one third candidate sample image, and the step of selecting at least one third candidate sample image similar to the current query image from the sample image set is continued until the at least one third candidate sample image contains a second type of first candidate sample image or the set number of queries is reached.

[0260] The second acquisition module updates the current query image based on the at least one third candidate sample image, and may include any one of the following:

[0261] The third candidate image with the highest similarity to the current query image among the at least one third candidate image is taken as the current query image;

[0262] Based on the similarity between each third candidate sample image in the at least one third candidate sample image and the current query image, the third candidate sample images are fused together, and the fused image is used as the current query image.

[0263] In another embodiment of this application, an electronic device is provided.

[0264] Reference Figure 10 Electronic devices may include:

[0265] The memory 100 is used to store computer programs.

[0266] Processor 200 is configured to execute the computer program to enable the electronic device to perform the following method steps:

[0267] Obtain the input image;

[0268] At least one first candidate sample image matching the input image is obtained from the sample image set; each of the at least one first candidate sample image has annotation information, which is used to distinguish at least two regions of the first candidate sample image; wherein, the first candidate sample image includes a sample image matching a second candidate sample image from the sample image set; the second candidate sample image matches the input image but does not have annotation information;

[0269] Based on the at least one first candidate sample image, the style of at least two regions of the input image is enhanced to obtain at least two enhanced regions;

[0270] A target image is generated based on at least two regions after the enhancement process.

[0271] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0272] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0273] In the above embodiments, the implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, in the form of a computer program product.

[0274] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. An image generation method, comprising: Obtain the input image; Obtain at least one first candidate sample image that matches the input image from the sample image set; Each of the at least one first candidate sample image has annotation information, which is used to distinguish at least two regions of the first candidate sample image; wherein, the first candidate sample image includes a sample image in the sample image set that matches the second candidate sample image; the second candidate sample image matches the input image and does not have annotation information; Based on the at least one first candidate sample image, the style of at least two regions of the input image is enhanced to obtain at least two enhanced regions; A target image is generated based on at least two regions after the enhancement process.

2. The image generation method according to claim 1, wherein obtaining at least one first candidate sample image matching the input image from the sample image set comprises: Obtain a first candidate sample image and / or a second candidate sample image of the first category that match the input image from the sample image set; The first candidate sample image of the first category has annotation information; The second candidate sample image does not have annotation information; The annotation information is used to distinguish at least two regions of the corresponding candidate sample image; Once the existence of the second candidate sample image is determined, a second type of first candidate sample image matching the second candidate sample image is obtained from the sample image set; the second type of first candidate sample image has annotation information.

3. The image generation method according to claim 2, wherein the step of enhancing the style of at least two regions of the input image based on the at least one first candidate sample image to obtain at least two enhanced regions includes: Based on the annotation information of the first candidate sample image of the first type and the first candidate sample image of the second type respectively, determine the style parameters corresponding to at least two regions of the first candidate sample image of the first type and the first candidate sample image of the second type respectively; Based on the style parameters corresponding to at least two regions of the second type of first candidate sample image, and the similarity between the second type of first candidate sample image and the second candidate sample image, the style parameters corresponding to at least two regions of the second candidate sample image are determined. Based on the style parameters corresponding to at least two regions of each of the first candidate sample image and the second candidate sample image, and the similarity between each of the first candidate sample image and the second candidate sample image and the input image, the style parameters corresponding to at least two regions of the input image are determined. Based on the style parameters corresponding to at least two regions of the input image, enhancement processing is performed on at least two regions of the input image to obtain at least two enhanced regions.

4. The image generation method according to claim 3, wherein there are multiple first candidate sample images of the second type; the step of determining the style parameters corresponding to at least two regions of the second candidate sample image based on the style parameters corresponding to at least two regions of the second candidate sample image and the similarity between the second candidate sample image and the second candidate sample image includes: Based on the similarity between each second-class first candidate sample image and the second candidate sample image, a first weight factor is determined for each second-class first candidate sample image. The first weighting factor characterizes the relative similarity between the second candidate sample image and the second class of first candidate sample images; Based on the first weighting factor, the style parameters corresponding to at least two regions of each of the second type of first candidate sample images are fused to obtain the style parameters corresponding to at least two regions of the second candidate sample image.

5. The image generation method according to claim 3, wherein determining the style parameters corresponding to at least two regions of the input image based on the style parameters corresponding to at least two regions of each of the first candidate sample image and the second candidate sample image, and the similarity between each of the first candidate sample image and the second candidate sample image and the input image, comprises: Based on the similarity between the first candidate sample image and the second candidate sample image and the input image, a second weight factor and a third weight factor are determined for the first candidate sample image and the second candidate sample image, respectively. The second weighting factor characterizes the relative similarity between the first candidate sample image of the first class and the input image; The third weighting factor characterizes the relative similarity between the second candidate sample image and the input image; Based on the second weighting factor and the third weighting factor, the style parameters corresponding to at least two regions of the first candidate sample image and the second candidate sample image are fused to obtain the style parameters corresponding to at least two regions of the input image.

6. The image generation method according to claim 3, wherein the enhancement processing of at least two regions in the input image based on style parameters corresponding to at least two regions of the input image to obtain at least two enhanced regions includes: Based on the style parameters corresponding to at least two regions of the input image, style normalization processing is performed on at least two regions of the input image. Based on the style parameters corresponding to at least two regions of the input image, style reconstruction processing is performed on the at least two regions after normalization to obtain at least two regions after enhancement processing.

7. The image generation method according to claim 1, wherein generating the target image based on the at least two regions after enhancement processing includes: Obtain the annotation information of the input image; The annotation information of the input image includes at least two regions used to distinguish the input image; Based on the annotation information of the input image, denoising processing is performed on at least two regions after the enhancement process; At least two regions after denoising are fused to obtain the target image.

8. The image generation method according to claim 2, wherein obtaining a second type of first candidate sample image matching the second candidate sample image from the sample image set comprises: Use the second candidate sample image as the current query image; Query at least one third candidate sample image from the set of sample images that is similar to the currently queried image; If none of the at least one third candidate sample image has annotation information, the current query image is updated based on the at least one third candidate sample image, and the step of selecting at least one third candidate sample image similar to the current query image from the sample image set is continued until the at least one third candidate sample image contains a second type of first candidate sample image or the set number of queries is reached.

9. The image generation method according to claim 8, wherein updating the current query image based on the at least one third candidate sample image includes any one of the following: The third candidate image with the highest similarity to the current query image among the at least one third candidate image is taken as the current query image; Based on the similarity between each third candidate sample image in the at least one third candidate sample image and the current query image, the third candidate sample images are fused together, and the fused image is used as the current query image.

10. An electronic device, comprising: Memory is used to store computer programs; The processor is used to execute the computer program to enable the electronic device to perform the following method steps: Obtain the input image; At least one first candidate sample image matching the input image is obtained from the sample image set; each of the at least one first candidate sample image has annotation information, which is used to distinguish at least two regions of the first candidate sample image; wherein, the first candidate sample image includes a sample image matching a second candidate sample image from the sample image set; the second candidate sample image matches the input image but does not have annotation information; Based on the at least one first candidate sample image, the style of at least two regions of the input image is enhanced to obtain at least two enhanced regions; A target image is generated based on at least two regions after the enhancement process.

Images