An AIGC-based interactive intelligent content generation method and system
By performing optical path geometry verification and synchronous rendering in the AIGC generation algorithm, the problem of light and shadow fragmentation in the existing technology is solved, achieving efficient image light and shadow processing and improving the realism and blending of the image.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- EDEN INFORMATION SERVICE LTD
- Filing Date
- 2026-03-06
- Publication Date
- 2026-06-23
Smart Images
Figure CN121810997B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence technology, specifically to an interactive intelligent content generation method and system based on AIGC. Background Technology
[0002] With the rapid development of AI-generated content (AIGC) technology, existing image editing tools typically allow users to perform color replacement, style transfer, or local redrawing of specific targets in images through natural language commands, lowering the threshold for professional image processing and enabling non-professional users to quickly produce high-quality visual content.
[0003] However, existing AIGC generation algorithms suffer from a technical flaw: a disconnect between semantic changes and the physical environment. Most mainstream models rely on statistical predictions based on pixel features in a two-dimensional plane, lacking explicit awareness of the scene's three-dimensional spatial structure and the laws of physical optical propagation. For example, when a user modifies an image by changing a subject to a bright or specific colored luminous material, traditional algorithms often only change the object's pixel values, ignoring the diffuse reflection, specular mapping, and light attenuation effects the luminous object should produce on the surrounding environment. This frequently results in erroneous generation of unreasonable lighting effects in occluded areas.
[0004] The lack of physical logic often results in generated images that look like obvious textures or collages, with a disconnect between the subject and the environment in terms of light and shadow, which undermines the overall realism and immersion of the image and fails to meet the design requirements of film and television or professional levels. In addition, in order to correct these lighting and shadow errors that do not meet the requirements, designers often need to use traditional graphics software for tedious manual post-processing, which weakens the automation and high efficiency advantages that AIGC technology should have.
[0005] Therefore, the present invention provides an interactive intelligent content generation method and system based on AIGC. Summary of the Invention
[0006] The purpose of this invention is to provide an interactive intelligent content generation method and system based on AIGC to solve the aforementioned background problems.
[0007] The objective of this invention can be achieved through the following technical solutions:
[0008] An interactive intelligent content generation method based on AIGC includes the following steps:
[0009] The first semantic subject is obtained as the index and an attention weight matrix for the entire image is established. High-response pixels are selected based on the attention weight matrix, and connectivity analysis is performed on the high-response pixels to determine candidate linkage regions.
[0010] Obtain the spatial structure information of the candidate linkage region, construct the optical path geometry verification logic based on the spatial structure information, and determine whether there is physical occlusion or self-occlusion effect between the first semantic subject and the candidate linkage region. If so, filter the effective response region.
[0011] The image statistical features of the first semantic subject are extracted and color semantic change processing is performed to obtain the target change vector;
[0012] Based on the spatial location distribution of the effective response region, spatial distance attenuation analysis is performed on the target change vector to obtain the basic attenuation coefficient; material scattering correction is performed on the basic attenuation coefficient by obtaining image statistical features, and the cooperative mapping parameters of the effective response region are derived.
[0013] A light and shadow guidance matrix is constructed based on the collaborative mapping parameters. The effective response area is then guided by the light and shadow guidance matrix to achieve synchronous rendering of the first semantic subject and the effective response area.
[0014] Furthermore, the method for obtaining the first semantic subject is as follows:
[0015] The system collects user editing instructions for the target image and performs intent parsing to obtain the first semantic subject in the target image.
[0016] Furthermore, the intent parsing is performed as follows:
[0017] The AIGC large model uses a pre-built multimodal semantic understanding unit to extract keywords from user editing commands and identify the action subject in the commands.
[0018] Text embedding vectors for user editing commands are built based on action subjects;
[0019] Simultaneously, the visual segmentation model is invoked to perform panoramic segmentation on the target image, obtaining the mask regions of all independent objects in the image;
[0020] Image embedding vectors are established based on the mask regions of independent objects;
[0021] The text embedding vector of the action subject is matched with the image embedding vectors of all independent objects using cosine similarity. The mask region with the highest similarity score is selected and defined as the first semantic subject.
[0022] Furthermore, the connectivity analysis is performed as follows:
[0023] A heatmap of full-graph attention is constructed and zero-weighted screening is performed. Based on the screening results, a discrete set of high-response points is constructed.
[0024] Density clustering algorithm is used to spatially cluster discrete high-response point sets, and candidate linkage regions are established based on the spatial clustering results.
[0025] Furthermore, the method for determining whether physical occlusion or self-occlusion effects exist is as follows:
[0026] Each pixel within the candidate linkage area is used as a target verification point;
[0027] Obtain the 3D surface normal map and ray propagation vector;
[0028] Extract the unit normal vector corresponding to the target verification point in the 3D surface normal map, perform reflection compatibility verification based on the normal angle on the target verification point, and remove target verification points with self-occlusion effect;
[0029] The light propagation vector is reprojected to obtain path sampling points, and depth difference comparison is performed to verify whether the path sampling points block the light propagation path; based on the verification results, the target verification points are eliminated.
[0030] Furthermore, the process of obtaining the light propagation vector is as follows:
[0031] Feature extraction is performed on the target image to generate a panoramic depth gradient map representing the relative distances between pixels;
[0032] Simultaneously, a three-dimensional surface normal map is established. Based on the principle of geometric inverse projection, and combining the two-dimensional coordinates of the image plane with the depth values in the panoramic depth gradient map, a pseudo-three-dimensional point cloud space is established.
[0033] Calculate the geometric center of the mask region corresponding to the first semantic subject in the pseudo-3D point cloud space, and use it as the virtual radiation source point; establish a straight line vector from the virtual radiation source point to the target verification point in the pseudo-3D point cloud space, and use it as the light propagation vector.
[0034] Furthermore, the method for performing the depth differential comparison and verification is as follows:
[0035] Read the pixel values at the corresponding locations of the path sampling points from the panoramic depth gradient map as the actual depth;
[0036] Based on the depth values of the virtual radiation source and the target verification point, the theoretical projection depth of the path sampling point on the line connecting the rays is calculated.
[0037] If at any path sampling point the actual depth is lower than the theoretical projection depth, then it is determined that there is a physical entity at the location corresponding to the path sampling point that is obstructing the path of light propagation.
[0038] If a physical entity obstructs the path of light propagation, it is determined that there is a physical occlusion between the first semantic subject and the target verification point, and the target verification point is removed from the candidate linkage area.
[0039] Furthermore, the method for performing the material scattering correction is as follows:
[0040] High-frequency texture feature values are extracted from the statistical features of the image and defined as the detail adjustment factor;
[0041] Damping correction logic is constructed based on detailed adjustment factors, and the basic attenuation coefficient is weighted down to obtain the comprehensive conduction coefficient.
[0042] The detail adjustment factor and the scattering sensitivity control constant are multiplied together, and the result of the multiplication is summed with the preset benchmark value to obtain the scattering damping denominator.
[0043] Calculate the ratio of the basic attenuation coefficient to the denominator of the scattering damping, and define the resulting ratio as the comprehensive conduction coefficient;
[0044] Keeping the vector direction of the target change vector unchanged, the vector magnitude of the target change vector is scaled and modulated using the comprehensive transmission coefficient to obtain the target magnitude;
[0045] The vector is reconstructed based on the vector direction and the target magnitude, and the reconstructed vector result is defined as the cooperative mapping parameter.
[0046] Furthermore, the basic attenuation coefficient is obtained as follows:
[0047] Squaring the spatial Euclidean distance of each pixel;
[0048] The result of the squaring operation is made dimensionless, and the dimensionless spatial Euclidean distance is summed with the preset constant illumination intensity to obtain the distance attenuation denominator.
[0049] Calculate the ratio of the maintenance constant of light intensity to the denominator of the distance attenuation, and define the ratio as the basic attenuation coefficient.
[0050] An interactive intelligent content generation system based on AIGC includes the following modules:
[0051] Intent parsing module: used to collect user editing instructions for the target image and perform intent parsing to obtain the first semantic subject in the target image; extract the image statistical features of the first semantic subject and perform color semantic change processing to obtain the target change vector;
[0052] Connectivity Analysis Module: Used to build an attention weight matrix for the entire graph with the first semantic subject as the index; based on the attention weight matrix, high-response pixels are selected, and connectivity analysis is performed on the high-response pixels to determine candidate linkage regions;
[0053] Region filtering module: used to obtain the spatial structure information of candidate linkage regions, construct optical path geometry verification logic based on the spatial structure information, and determine whether there is physical occlusion or self-occlusion effect between the first semantic subject and the candidate linkage region. If so, the effective response region is filtered.
[0054] Attenuation correction module: Based on the spatial location distribution of the effective response region, the target change vector is analyzed for spatial distance attenuation to obtain the basic attenuation coefficient; based on the image statistical features of the first semantic subject, the basic attenuation coefficient is corrected for material scattering, and the cooperative mapping parameters of the effective response region are derived.
[0055] Synchronous rendering module: Constructs a light and shadow guidance matrix based on collaborative mapping parameters, and performs directional guidance processing on the effective response area based on the light and shadow guidance matrix to achieve synchronous rendering of the first semantic subject and the effective response area.
[0056] The beneficial effects of this invention are as follows:
[0057] 1. By adopting a subject localization strategy that combines AIGC semantic understanding and visual segmentation technology, it is beneficial to separate the physical anchor points corresponding to the user's intent from the image background. By combining HSV color space conversion and statistical analysis, when obtaining the target change vector, the brightness distribution variance and high-frequency texture features of the first semantic subject can be preserved. This allows the original material reflective properties and surface roughness details of the object to be maintained in the subsequent color semantic change process, thereby improving the degree of preservation of the object's intrinsic physical attributes during content generation.
[0058] 2. By constructing a weight matrix using the self-attention mechanism of the generative model decoder, regions in the image that have potential visual associations with the first semantic subject can be identified. Combined with the DBSCAN density clustering algorithm, spatial constraints are applied to discrete high-response point sets. This helps to filter out discrete isolated noise points generated during the attention calculation process of the generative model, and confines the linked regions to a sheet-like range with density connectivity. This allows for the selection of logically related potential reflection or illumination-affected regions in complex backgrounds, reducing the erroneous modification rate of irrelevant background regions during image generation.
[0059] 3. By calculating the basic attenuation coefficient and the comprehensive conduction coefficient, the target change vector is modulated so that the response intensity of the linkage area decreases linearly with the increase of distance and the scattering weakens with the increase of material roughness. This replicates the physical light attenuation effect in image generation and improves the problem of uniform light and shadow distribution and lack of layering in traditional generation methods.
[0060] 4. Construct a light and shadow guidance matrix and inject it as a bias term into the iterative denoising process. Combined with a structure locking strategy for high-frequency information, it can suppress changes in image edge lines and texture particles while adjusting the color and lighting atmosphere of the effective response area. Through directional guidance processing, the first semantic subject and its environmental lighting effects are rendered synchronously, which not only achieves the expected visual changes but also maintains the original structural integrity and texture details of the image. This solves the structural collapse or texture blurring that easily occurs during image redrawing and improves the image fusion and visual harmony of the final generated image. Attached Figure Description
[0061] The invention will now be further described with reference to the accompanying drawings.
[0062] Figure 1 This is a flowchart of an interactive intelligent content generation method based on AIGC according to the present invention;
[0063] Figure 2 This is a flowchart of the present invention for determining the existence of a self-occlusion effect;
[0064] Figure 3 This is a functional block diagram of an AIGC-based interactive intelligent content generation system in this invention. Detailed Implementation
[0065] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below in conjunction with specific embodiments.
[0066] Example 1
[0067] like Figure 1 As shown, an interactive intelligent content generation method based on AIGC includes the following steps:
[0068] S10. Collect user editing instructions for the target image and perform intent parsing to obtain the first semantic subject in the target image; extract the image statistical features of the first semantic subject and perform color semantic change processing to obtain the target change vector;
[0069] The method for obtaining the first semantic subject in the target image by collecting the user's editing instructions on the target image and performing intent parsing is as follows:
[0070] In some preferred embodiments, the multimodal semantic understanding unit pre-built in the AIGC large model extracts keywords from user editing instructions and identifies the action subject in the instructions;
[0071] Text embedding vectors for user editing commands are built based on action subjects;
[0072] Simultaneously, the visual segmentation model is invoked to perform panoramic segmentation on the target image, obtaining the mask regions of all independent objects in the image;
[0073] Image embedding vectors are established based on the mask regions of independent objects;
[0074] Those skilled in the art will understand that the text embedding vector is processed by segmenting and serializing the identified action subject, inputting it into the AIGC pre-trained text encoder to extract deep semantic features, and then mapping it to a joint feature space of a preset dimension through a linear projection layer before performing L2 normalization.
[0075] The image embedding vector is cropped from the target image based on the boundary coordinates of the mask region. After size normalization preprocessing, it is input into the AIGC visual encoder to extract global visual features. Similarly, it is mapped to the joint feature space through linear projection and L2 normalization. That is, the text and image vectors are mathematically comparable in the same feature space.
[0076] The text embedding vector of the action subject is matched with the image embedding vectors of all independent objects using cosine similarity. The mask region with the highest similarity score is selected and defined as the first semantic subject.
[0077] It should be noted that the first semantic subject is the only physical anchor point for subsequent image redrawing operations, the mask area is used to determine the spatial boundary of pixel modification, and the algorithm only applies to the user-specified target and will not accidentally damage other objects of similar color in the image background.
[0078] The method for extracting the image statistical features of the first semantic subject and performing color semantic change processing to obtain the target change vector is as follows:
[0079] Preferably, the pixel area corresponding to the first semantic subject is converted from the RGB color space to the HSV (hue, saturation, brightness) color space;
[0080] Statistical analysis was performed on the pixel regions of the converted color space to calculate the mean hue, variance of brightness distribution, and high-frequency texture feature values within the regions.
[0081] The hue mean, luminance distribution variance, and high-frequency texture feature values are encapsulated into a feature tensor as image statistical features;
[0082] It should be noted that image statistical features represent the initial physical state of an object before it is modified. The variance of the brightness distribution of image statistical features characterizes the intensity of reflection on the object's surface (i.e., the smoothness of the material), and the high-frequency texture feature value characterizes the roughness of the object's surface.
[0083] Preferably, adjective descriptive words (such as "blue" or "glowing") are extracted from user editing instructions, and a color-semantic mapping table pre-stored in the model is read.
[0084] It should be noted that the color-semantic mapping table is established during the model training phase by analyzing a large number of text-image pairs (i.e., the binding combination established between adjective descriptive words and images) to create a lookup table that corresponds color descriptive words in natural language to numerical ranges in the HSV space.
[0085] Substitute the adjective descriptive words into the color-semantic mapping table for indexing to obtain the target HSV value expected by the user.
[0086] The target HSV value is calculated by subtracting the mean hue and mean brightness values in the image statistical features. The resulting combination of differences is the target change vector.
[0087] It is understandable that the target HSV value contains hue and brightness values, and the target change vector is mathematically represented as a vector with direction and magnitude.
[0088] S20. Establish an attention weight matrix for the entire image using the first semantic subject as the index; filter high-response pixels based on the attention weight matrix, and perform connectivity analysis on the high-response pixels to determine candidate linkage regions;
[0089] The attention weight matrix for the entire image is established using the first semantic subject as an index; the method for selecting high-response pixels based on the attention weight matrix is as follows:
[0090] Preferably, feature layers with a preset resolution (such as 32×32 or 64×64) are extracted from the AIGC image generation model decoder;
[0091] The original mask of the first semantic subject is downsampled to the same resolution as the feature layer to obtain the index mask at the feature layer scale;
[0092] It should be noted that the reason for downsampling is that the original mask is usually based on the original image resolution (such as 1024×1024), while the self-attention calculation of the AIGC model is performed in the compressed feature space, and spatial dimension alignment is necessary for accurate correspondence.
[0093] The self-attention tensor of the feature layer is truncated by the forward hook mechanism. The attention weight values of the first semantic subject to other pixels of the whole image are extracted from the self-attention tensor by the index mask and combined to form a heatmap of the whole image attention.
[0094] The method for determining candidate linkage regions by performing connectivity analysis on high-response pixels is as follows:
[0095] Gaussian smoothing and noise reduction are applied to the heatmap of attention across the entire image, while a preset linkage response threshold is read.
[0096] Preferably, the linkage response threshold is 80% of the quantile of the full-image attention heatmap;
[0097] Reset the attention weight of pixels in the heatmap whose values are lower than or equal to the linkage response threshold to zero;
[0098] Filter pixels with attention weights higher than a threshold for linkage response to establish a discrete set of high-response points;
[0099] The density clustering algorithm of DBSCAN is used to perform spatial clustering on discrete high-response point sets, and candidate linkage regions are established based on the spatial clustering results.
[0100] Those skilled in the art will understand that the method for spatial clustering is as follows: by setting a minimum number of cluster points and a search radius, isolated noise clusters are eliminated, connected pixel regions that meet the density requirements are merged, and the original image resolution is restored by upsampling to obtain candidate linked regions;
[0101] The physical significance of density clustering analysis is that real light and shadow reflections (such as reflections) are usually continuous sheet-like regions in space, while the errors generated by the AIGC model calculation are usually scattered isolated noise points. Density and connectivity constraints can effectively filter out misjudged regions.
[0102] The process of defining candidate linkage regions is similar to throwing a stone into a calm water surface (the first semantic subject). The affected area is determined by capturing the range of ripples generated on the water surface. Only areas that are on the ripple transmission path and have continuous ripples are determined to be linkage regions that need to follow the changes of the subject.
[0103] Example 2
[0104] Please see Figure 1 As shown, an interactive intelligent content generation method based on AIGC includes the following steps:
[0105] S30. Obtain the spatial structure information of the candidate linkage region, construct the optical path geometry verification logic based on the spatial structure information, and determine whether there is physical occlusion or self-occlusion effect between the first semantic subject and the candidate linkage region. If so, filter the effective response region.
[0106] The method for obtaining spatial structure information of candidate linkage regions and constructing optical path geometry verification logic based on spatial structure information is as follows:
[0107] Preferably, a monocular depth estimation algorithm is used to extract features from the target image to generate a panoramic depth gradient map representing the relative distance between pixels;
[0108] Simultaneously, the spatial orientation of each pixel in the target image is calculated using a surface normal extraction algorithm to generate a three-dimensional surface normal map.
[0109] The three-dimensional surface normal map and the panoramic depth gradient map are used as spatial structural information of the candidate linkage region.
[0110] It should be noted that the pixel grayscale values in the panoramic depth gradient map have a monotonic mapping relationship with the distance from the object to the viewpoint, and the RGB channel values in the 3D surface normal map correspond to the X, Y, and Z axis components of the normal vector in the spatial coordinate system, respectively.
[0111] Based on the principle of geometric inverse projection, a pseudo-3D point cloud space is established by combining the two-dimensional coordinates of the image plane with the depth value in the panoramic depth gradient map.
[0112] In some embodiments, when the user editing instructions contain a semantic description characterizing light emission, enhanced brightness, or significant changes in illumination, the first semantic subject is considered to possess characteristics of light effect propagation;
[0113] The geometric center of the first semantic subject in the pseudo-3D point cloud space is defined as the virtual radiation source point, which is used to construct the spatial influence propagation path.
[0114] When the user's editing instructions do not contain semantic features, the virtual radiation source point is only used as a reference anchor point for the propagation of intensity, and does not represent a real physical light source.
[0115] Each pixel within the candidate linkage area is used as a target verification point;
[0116] In the pseudo-3D point cloud space, a straight line vector from the virtual radiation source point to the target verification point is established as the light propagation vector.
[0117] It is understandable that the purpose of constructing a pseudo-3D point cloud space is:
[0118] Objective 1: To convert the visual relationships of a two-dimensional image plane into geometric relationships in three-dimensional space;
[0119] Objective 2: A unified inverse projection transformation can improve the accuracy of occlusion and angular relationships in target images;
[0120] The method for determining whether there is physical occlusion or self-occlusion effect between the first semantic subject and the candidate linkage region based on optical path geometric verification logic is as follows:
[0121] S301. Extract the unit normal vector corresponding to the target verification point in the three-dimensional surface normal map, perform reflection compatibility verification based on the normal angle on the target verification point, and remove target verification points with self-occlusion effect.
[0122] The method for performing reflection compatibility verification based on the included normal angle is as follows:
[0123] Calculate the dot product of the ray propagation vector and the unit normal vector corresponding to the target verification point, and use it as the light reception determination value;
[0124] Set a light reception threshold; preferably, set it to 0.1.
[0125] like Figure 2 As shown, when the light-receiving judgment value is less than the light-receiving judgment threshold, it indicates that the surface of the target verification point is facing away from the virtual radiation source point. The target verification point is judged to have a self-occlusion effect and is removed from the candidate linkage area.
[0126] When the light-receiving judgment value is greater than or equal to the threshold, it indicates that the target verification point meets the light-receiving conditions, and the target verification point is retained for subsequent processing. If the light-receiving judgment value is lower than the light-receiving judgment threshold, the light-receiving judgment value is continuously monitored.
[0127] S302. Reproject the light propagation vector to obtain path sampling points and perform depth difference comparison verification to check whether the path sampling points block the light propagation path; based on the verification results, remove the target verification points.
[0128] Preferably, the reprojection process is performed by projecting the light propagation vector onto a two-dimensional image plane to obtain a two-dimensional projection path;
[0129] Discretize the two-dimensional projection path to obtain N path sampling points;
[0130] The method for performing depth differential comparison and verification is as follows:
[0131] Read the pixel values at the corresponding locations of the path sampling points from the panoramic depth gradient map as the actual depth;
[0132] Based on the depth values of the virtual radiation source and the target verification point, a linear interpolation algorithm is used to calculate the theoretical projection depth of the path sampling point on the ray line;
[0133] If the actual depth is lower than the theoretical projection depth at any path sampling point, it is determined that there is a physical entity obstructing the light propagation path at the location corresponding to the path sampling point; otherwise, no action is taken.
[0134] If a physical entity obstructs the path of light propagation, it means that there is a physical occlusion between the first semantic subject and the target verification point, and the target verification point will be removed from the candidate linkage area.
[0135] Obtain the candidate linkage regions that have not undergone the elimination process, and use them as valid response regions.
[0136] S40. Based on the spatial location distribution of the effective response region, perform spatial distance attenuation analysis on the target change vector to obtain the basic attenuation coefficient; based on the image statistical features of the first semantic subject, perform material scattering correction on the basic attenuation coefficient to derive the collaborative mapping parameters of the effective response region.
[0137] The method for obtaining the basic attenuation coefficient by performing spatial distance attenuation analysis on the target change vector based on the spatial location distribution of the effective response area is as follows:
[0138] Preferably, based on the constructed pseudo-3D point cloud space, the spatial Euclidean distance of each pixel within the effective response area relative to the virtual radiation source point is calculated;
[0139] By performing an inverse mapping between spatial Euclidean distance and energy intensity, the basic attenuation coefficient of the pixel is obtained.
[0140] Preferably, the method for performing inverse mapping is as follows:
[0141] Set a maintenance constant for the light intensity;
[0142] It should be noted that the purpose of setting the maintenance constant for light intensity is to prevent numerical singularities from occurring when the distance is zero;
[0143] The spatial Euclidean distance in the pseudo-3D point cloud space is normalized. The normalization process is calculated based on the maximum spatial scale of the effective response region in the target image (i.e., the square of the maximum spatial Euclidean distance).
[0144] The dimensionless spatial Euclidean distance is squared and summed with a preset dimensionless illumination intensity constant to obtain the distance attenuation denominator.
[0145] Preferably, the value range of the light intensity maintenance constant C is [0.01, 1.0], with a preferred value of 0.1, to prevent calculation overflow when the distance is zero;
[0146] Calculate the ratio of the illumination intensity maintenance constant to the distance attenuation denominator, and define the resulting ratio as the basic attenuation coefficient;
[0147] Those skilled in the art will understand that as the spatial Euclidean distance increases, the distance attenuation denominator will increase quadratically, causing the final calculated basic attenuation coefficient (i.e., ratio) to drop rapidly and approach zero infinitely, thus replicating the natural attenuation effect of light propagation in the real world.
[0148] The method for deriving the co-mapping parameters of the effective response region by correcting the material scattering of the basic attenuation coefficient based on the image statistical features of the first semantic subject is as follows:
[0149] Preferably, high-frequency texture feature values are extracted from the statistical features of the image;
[0150] It should be noted that high-frequency texture feature values are used to characterize the complexity of surface details of the first semantic subject and are defined as detail adjustment factors;
[0151] The surface detail adjustment factor is used to empirically modulate the propagation intensity of the target change vector during the generation process. Its role is to simulate the effect of the complexity of surface details on the perceived changes in illumination. The surface detail adjustment factor is a generation control parameter and does not directly correspond to the real physical diffuse reflection parameter.
[0152] Damping correction logic is constructed based on detailed adjustment factors, and the basic attenuation coefficient is weighted down to obtain the comprehensive conduction coefficient.
[0153] The preferred method for weighted down adjustment is as follows:
[0154] Set a scattering sensitivity tuning constant;
[0155] It should be noted that the purpose of setting the scattering sensitivity adjustment constant is to: standardize and constrain the numerical magnitude of high-frequency texture feature values, adjust the influence weight of surface roughness on light energy attenuation, and prevent the calculation results from being distorted due to excessive fluctuations in the original feature values.
[0156] The detail adjustment factor and the scattering sensitivity control constant are multiplied together, and the result of the multiplication is summed with the preset reference value (the preset reference value is 1) to obtain the scattering damping denominator.
[0157] Calculate the ratio of the basic attenuation coefficient to the denominator of the scattering damping, and define the resulting ratio as the comprehensive conduction coefficient;
[0158] Keeping the vector direction of the target change vector unchanged, the vector magnitude of the target change vector is scaled and modulated using the comprehensive transmission coefficient to obtain the target magnitude;
[0159] Preferably, the scaling modulation method is as follows: obtain the original magnitude of the target change vector, calculate the product of the original magnitude and the comprehensive transmission coefficient to obtain the target magnitude;
[0160] The reconstructed vector is based on the vector direction and the target magnitude, and the reconstructed vector result is defined as the cooperative mapping parameter;
[0161] As can be understood by those skilled in the art, the co-mapping parameter is essentially a construction instruction with spatial and material constraints. The physical meaning of the co-mapping parameter is: under the premise of keeping the hue direction of color change consistent, the actual brightness or color change that should occur at the point is calculated based on the distance of the point from the light source (reflected by the base attenuation coefficient) and the roughness of the reflective surface (reflected by the scattering damping denominator).
[0162] S50. Construct a light and shadow guidance matrix based on the collaborative mapping parameters, and perform directional guidance processing on the effective response area based on the light and shadow guidance matrix to achieve synchronous rendering of the first semantic subject and the effective response area.
[0163] The method for constructing the light and shadow guidance matrix based on the cooperative mapping parameters is as follows:
[0164] Preferably, the collaborative mapping parameters are arranged according to their coordinate positions in the image to construct a light and shadow guidance matrix aligned with the target image resolution, and the iterative denoising process of the AIGC image generation model is initiated.
[0165] It should be noted that the iterative denoising process is the core mechanism of AIGC image generation, similar to a painter starting from a blank sheet of paper (pure noise) and gradually creating a clear image through dozens of brushstrokes (denoising steps).
[0166] The method for synchronous rendering of the first semantic subject and the effective response area by using a light and shadow guidance matrix to perform directional guidance processing on the effective response area is as follows:
[0167] In each intermediate computation step of the iterative denoising process, the light and shadow guidance matrix is input into the pre-trained variational autoencoder for dimensionality reduction encoding to obtain the latent guidance tensor aligned with the dimension of the noise prediction result.
[0168] After multiplying the tensor by a preset guiding strength coefficient, it is added to the noise prediction residual of the diffusion model by adding them element by element.
[0169] The preferred method for superposition is as follows:
[0170] Obtain the original pixel values currently predicted by the model and read the parameter values at the corresponding positions in the light and shadow guidance matrix;
[0171] The parameter values are applied to the original pixel values in an incremental manner, causing the pixel values to shift towards the target hue and brightness, thereby achieving directional guidance processing;
[0172] The method for achieving synchronized visual rendering of the first semantic subject and the effective response area while preserving texture details is as follows:
[0173] While injecting the light and shadow guidance matrix, a structure locking strategy is executed on the effective response area;
[0174] Preferably, the structure locking strategy is implemented as follows: extract the latent spatial feature map of the previous generation step, separate the high-frequency edge components through a high-pass filter; in the noise prediction stage of the current generation step, inject the high-frequency edge components into the corresponding skip connection layer of the U-Net decoder in the form of residual connections to constrain the smearing range of low-frequency light and shadow.
[0175] Example 3
[0176] Please see Figure 3 As shown, an interactive intelligent content generation system based on AIGC includes the following modules:
[0177] Intent parsing module: used to collect user editing instructions for the target image and perform intent parsing to obtain the first semantic subject in the target image; extract the image statistical features of the first semantic subject and perform color semantic change processing to obtain the target change vector;
[0178] Connectivity Analysis Module: Used to build an attention weight matrix for the entire graph with the first semantic subject as the index; based on the attention weight matrix, high-response pixels are selected, and connectivity analysis is performed on the high-response pixels to determine candidate linkage regions;
[0179] Region filtering module: used to obtain the spatial structure information of candidate linkage regions, construct optical path geometry verification logic based on the spatial structure information, and determine whether there is physical occlusion or self-occlusion effect between the first semantic subject and the candidate linkage region. If so, the effective response region is filtered.
[0180] Attenuation correction module: Based on the spatial location distribution of the effective response region, the target change vector is analyzed for spatial distance attenuation to obtain the basic attenuation coefficient; based on the image statistical features of the first semantic subject, the basic attenuation coefficient is corrected for material scattering, and the cooperative mapping parameters of the effective response region are derived.
[0181] Synchronous rendering module: Constructs a light and shadow guidance matrix based on collaborative mapping parameters, and performs directional guidance processing on the effective response area based on the light and shadow guidance matrix to achieve synchronous rendering of the first semantic subject and the effective response area.
[0182] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the present invention should still fall within the scope of the present invention.
Claims
1. An interactive intelligent content generation method based on AIGC, characterized in that, Includes the following steps: The first semantic subject is obtained as the index and an attention weight matrix for the entire image is established. High-response pixels are selected based on the attention weight matrix, and connectivity analysis is performed on the high-response pixels to determine candidate linkage regions. Obtain the spatial structure information of the candidate linkage region, construct the optical path geometry verification logic based on the spatial structure information, and determine whether there is physical occlusion or self-occlusion effect between the first semantic subject and the candidate linkage region. If so, filter the effective response region. The method for determining whether physical occlusion or self-occlusion exists is as follows: Each pixel within the candidate linkage area is used as a target verification point; Obtain the 3D surface normal map and ray propagation vector; Extract the unit normal vector corresponding to the target verification point in the 3D surface normal map, perform reflection compatibility verification based on the normal angle on the target verification point, and remove target verification points with self-occlusion effect; The light propagation vector is reprojected to obtain path sampling points, and depth difference comparison is performed to verify whether the path sampling points block the light propagation path; based on the verification results, target verification points are eliminated. The image statistical features of the first semantic subject are extracted and color semantic change processing is performed to obtain the target change vector; Based on the spatial location distribution of the effective response area, a spatial distance attenuation analysis is performed on the target change vector to obtain the basic attenuation coefficient; Image statistical features are obtained to correct the material scattering of the basic attenuation coefficient, and the co-mapping parameters of the effective response region are derived. A light and shadow guidance matrix is constructed based on the collaborative mapping parameters. The effective response area is then guided by the light and shadow guidance matrix to achieve synchronous rendering of the first semantic subject and the effective response area.
2. The interactive intelligent content generation method based on AIGC according to claim 1, characterized in that: The first semantic subject is obtained by: collecting the user's editing instructions on the target image and performing intent parsing to obtain the first semantic subject in the target image.
3. The interactive intelligent content generation method based on AIGC according to claim 2, characterized in that: The intent parsing is performed as follows: The AIGC large model uses a pre-built multimodal semantic understanding unit to extract keywords from user editing commands and identify the action subject in the commands. Text embedding vectors for user editing commands are built based on action subjects; Simultaneously, the visual segmentation model is invoked to perform panoramic segmentation on the target image, obtaining the mask regions of all independent objects in the image; Image embedding vectors are established based on the mask regions of independent objects; The text embedding vector of the action subject is matched with the image embedding vectors of all independent objects using cosine similarity. The mask region with the highest similarity score is selected and defined as the first semantic subject.
4. The interactive intelligent content generation method based on AIGC according to claim 1, characterized in that: The connectivity analysis is performed as follows: A heatmap of full-graph attention is constructed and zero-weighted screening is performed. Based on the screening results, a discrete set of high-response points is constructed. Density clustering algorithm is used to spatially cluster discrete high-response point sets, and candidate linkage regions are established based on the spatial clustering results.
5. The interactive intelligent content generation method based on AIGC according to claim 1, characterized in that: The process of obtaining the light propagation vector is as follows: Feature extraction is performed on the target image to generate a panoramic depth gradient map representing the relative distances between pixels; Simultaneously, a three-dimensional surface normal map is established. Based on the principle of geometric inverse projection, and combining the two-dimensional coordinates of the image plane with the depth values in the panoramic depth gradient map, a pseudo-three-dimensional point cloud space is established. Calculate the geometric center of the mask region corresponding to the first semantic subject in the pseudo-3D point cloud space, and use it as the virtual radiation source point; A virtual radiation source point to the target verification point is established in the pseudo-3D point cloud space as a straight line vector, which serves as the light propagation vector.
6. The interactive intelligent content generation method based on AIGC according to claim 1, characterized in that: The method for performing the depth differential comparison and verification is as follows: The pixel values of the corresponding positions of the path sampling points are read from the panoramic depth gradient map as the actual depth; the theoretical projection depth of the path sampling points on the line of light is calculated based on the depth values of the virtual radiation source point and the target verification point. If at any path sampling point the actual depth is lower than the theoretical projection depth, then it is determined that there is a physical entity at the location corresponding to the path sampling point that is obstructing the path of light propagation. If a physical entity obstructs the path of light propagation, it is determined that there is a physical occlusion between the first semantic subject and the target verification point, and the target verification point is removed from the candidate linkage area.
7. The interactive intelligent content generation method based on AIGC according to claim 1, characterized in that: The method for performing the material scattering correction is as follows: High-frequency texture feature values are extracted from the statistical features of the image and defined as the detail adjustment factor; Damping correction logic is constructed based on detailed adjustment factors, and the basic attenuation coefficient is weighted down to obtain the comprehensive conduction coefficient. The detail adjustment factor and the scattering sensitivity control constant are multiplied together, and the result of the multiplication is summed with the preset benchmark value to obtain the scattering damping denominator. Calculate the ratio of the basic attenuation coefficient to the denominator of the scattering damping, and define the resulting ratio as the comprehensive conduction coefficient; Keeping the vector direction of the target change vector unchanged, the vector magnitude of the target change vector is scaled and modulated using the comprehensive transmission coefficient to obtain the target magnitude; The vector is reconstructed based on the vector direction and the target magnitude, and the reconstructed vector result is defined as the cooperative mapping parameter.
8. The interactive intelligent content generation method based on AIGC according to claim 7, characterized in that: The basic attenuation coefficient is obtained as follows: Squaring the spatial Euclidean distance of each pixel; The result of the squaring operation is made dimensionless, and the dimensionless spatial Euclidean distance is summed with the preset constant illumination intensity to obtain the distance attenuation denominator. Calculate the ratio of the maintenance constant of light intensity to the denominator of the distance attenuation, and define the ratio as the basic attenuation coefficient.
9. An AIGC-based interactive intelligent content generation system, used to implement the AIGC-based interactive intelligent content generation method of any one of claims 1-8, characterized in that, Includes the following modules: Intent parsing module: used to collect user editing instructions for the target image and perform intent parsing to obtain the first semantic subject in the target image; extract the image statistical features of the first semantic subject and perform color semantic change processing to obtain the target change vector; Connectivity Analysis Module: Used to build the attention weight matrix of the entire graph using the first semantic subject as an index; High-response pixels are selected based on the attention weight matrix, and connectivity analysis is performed on the high-response pixels to determine candidate linkage regions. Region filtering module: used to obtain the spatial structure information of candidate linkage regions, construct optical path geometry verification logic based on the spatial structure information, and determine whether there is physical occlusion or self-occlusion effect between the first semantic subject and the candidate linkage region. If so, the effective response region is filtered. The method for determining whether physical occlusion or self-occlusion exists is as follows: Each pixel within the candidate linkage area is used as a target verification point; Obtain the 3D surface normal map and ray propagation vector; Extract the unit normal vector corresponding to the target verification point in the 3D surface normal map, perform reflection compatibility verification based on the normal angle on the target verification point, and remove target verification points with self-occlusion effect; The light propagation vector is reprojected to obtain path sampling points, and depth difference comparison is performed to verify whether the path sampling points block the light propagation path; based on the verification results, target verification points are eliminated. Attenuation correction module: Based on the spatial location distribution of the effective response area, it performs spatial distance attenuation analysis on the target change vector to obtain the basic attenuation coefficient; Based on the image statistical features of the first semantic subject, the basic attenuation coefficient is corrected by material scattering, and the cooperative mapping parameters of the effective response region are derived. Synchronous rendering module: Constructs a light and shadow guidance matrix based on collaborative mapping parameters, and performs directional guidance processing on the effective response area based on the light and shadow guidance matrix to achieve synchronous rendering of the first semantic subject and the effective response area.