Artistic creation method and system based on AI image generation and multi-medium printing

Abstract

The application provides a cultural and creative production method and system based on AI image generation and multi-medium printing, relates to the technical field of digital cultural and creative production, and comprises the following steps: generating a target image by fusing text semantics and sketch style, and planning a curved surface adaptive printing track and differential ink amount control according to a medium three-dimensional contour and material parameters, and finally completing printing. The integrated and personalized production from creative input to physical products is realized, and the artistic expression and printing precision of cultural and creative products on a complex medium surface are improved.

Description

Technical Field

[0001] This invention relates to the field of digital cultural and creative product production technology, and in particular to a method and system for producing cultural and creative products based on AI image generation and multi-media printing. Background Technology

[0002] In the field of cultural and creative product manufacturing, existing technologies typically rely on relatively independent image generation and physical production processes. A common practice is for designers to first complete a two-dimensional design using image generation tools or by hand, and then send the designed 2D image file to printing equipment for output. In this process, the image generation stage is often based on a single text description or style reference, and the generated result often fails to accurately incorporate the user's complex creative intent. In the printing stage, existing technologies are primarily designed for standard flat media, using preset, uniform printing parameters to map the image onto the media surface.

[0003] The conventional practices described above in existing technologies have significant drawbacks. On the one hand, in the creative expression stage, the integration of textual descriptions and visual references is often abrupt or involves only simple feature splicing, making it difficult to achieve both semantic accuracy and stylistic consistency in the generated images. This hinders deep cross-modal intent understanding and collaborative control, potentially causing the final cultural and creative designs to deviate from the user's original composite concept. On the other hand, in the physical implementation stage, directly applying images designed for flat surfaces to cultural and creative media with complex curved surfaces or special materials can lead to image distortion on the curved surfaces, and color reproduction can deviate due to differences in material absorption characteristics. Uniform printing parameters cannot adapt to variations in surface geometry and differences in media materials, severely impacting the visual fidelity and artistic expression of the final cultural and creative products on three-dimensional carriers. Summary of the Invention

[0004] The embodiments of the present invention provide a method and system for creating cultural and creative products based on AI image generation and multi-media printing, which can solve the problems in the prior art.

[0005] A first aspect of the present invention provides a method for creating cultural and creative products based on AI image generation and multi-media printing, comprising:

[0006] Obtain the user-input topic semantic text description and reference style sketch, convert the topic semantic text description into a semantic feature vector through a text encoder, and extract the style feature vector from the reference style sketch through a visual encoder;

[0007] The semantic feature vector and the style feature vector are fused across modal attention to generate a cross-modal joint guidance vector. The cross-modal joint guidance vector is used to apply joint constraints to the image generation model to generate a target image controlled by text semantics and visual style.

[0008] The target image is segmented into salient regions, and the color features and edge contour features of each salient region are extracted to form a partitioned visual feature representation. The surface three-dimensional contour point cloud data and material response parameters of the target medium are collected according to the target medium type identifier.

[0009] Based on the visual feature representation of the partition and the point cloud data of the surface three-dimensional contour, the salient region of the target image is projected and mapped onto the curvature field of the medium surface, an adaptive printhead motion trajectory for the curved medium is planned, and a partition-differentiated ink volume injection instruction set is generated in combination with the material response parameters.

[0010] Based on the partitioned differentiated ink volume injection instruction set, the target medium is subjected to surface adaptive printing output to complete the production of cultural and creative products.

[0011] The topic semantic text description is converted into a semantic feature vector using a text encoder, and the reference style sketch is used to extract a style feature vector using a visual encoder, including:

[0012] Semantic role labeling is performed on the topic semantic text description, the text content is decomposed into topic core word groups and style modification word groups, the topic core word groups and the style modification word groups are independently semantically modeled through the dual-path encoding sub-network of the text encoder, the fusion weights are assigned according to the semantic contribution of each encoding vector and weighted aggregation is performed, and the semantic feature vector that integrates topic and style information is output.

[0013] Superpixel segmentation is performed on the reference style sketch to obtain several semantically coherent superpixel regions. The pixel features in each superpixel region are averaged and pooled to obtain superpixel-level style descriptors. A region adjacency graph is constructed based on the spatial adjacency relationship between superpixels. The superpixel-level style descriptors are used as node features and input into the graph convolution module in the visual encoder. The graph convolution module propagates and aggregates the style information of adjacent regions layer by layer along the adjacent edges. The global readout layer of the visual encoder gathers the node features and outputs the style feature vector of the perceived region structure.

[0014] The semantic feature vector and the style feature vector are fused across modal attention to generate a cross-modal joint guidance vector. This cross-modal joint guidance vector is then used to apply joint constraints to the image generation model, generating a target image that is co-controlled by textual semantics and visual style.

[0015] Several semantic prototype vectors are extracted from the semantic feature vectors, each of which represents the main distribution direction of the semantic feature vectors in a semantic subspace; the cross-modal response values ​​between the style feature vectors and each semantic prototype vector are calculated using the semantic prototype vectors as anchor points; selective activation is performed on the style feature vectors based on the cross-modal response values; and the style activation vector anchored in the semantic subspace is output.

[0016] The style activation vector and the semantic feature vector are subjected to bidirectional cross-residual fusion. During the fusion process, the topic-dominant component of the semantic feature vector and the style-dominant component of the style activation vector are preserved. The fusion result is projected to a unified representation space through a feature mapping layer to output a cross-modal joint guiding vector.

[0017] The cross-modal joint guiding vector is decomposed into semantic constraint components and style constraint components. The semantic constraint components are injected into the semantic control path of the image generation model, and the style constraint components are injected into the style control path of the image generation model. The semantic control path and the style control path are fused collaboratively at the feature integration nodes of each layer. The image generation model performs multiple rounds of iterative sampling under the dual-path collaborative constraint, and the sampling results are collected to form the target image.

[0018] The semantic control pathway and style control pathway are collaboratively fused at each layer's feature integration node. The image generation model performs multiple rounds of iterative sampling under dual-path collaborative constraints, and the sampling results are aggregated to form the target image, including:

[0019] At each feature integration node, cross-path difference measurement is performed on the semantic layer features output by the semantic control path and the style layer features output by the style control path. The difference measurement result drives the difference compensation fusion, outputting the current layer integrated feature. The current layer integrated feature is superimposed on the subsequent layer input of the two paths in a residual manner. After each layer is completed in hierarchical order, the integrated features of each layer are weighted and aggregated according to the hierarchical weight to form a collaborative constraint signal.

[0020] The collaborative constraint signal is injected into the image generation model to form a dual-path collaborative constraint. In the first round of iterative sampling, the sampling constraint potential is initialized with the integrated features of each layer in the collaborative constraint signal to drive the first round of sampling and obtain the first round of sampling results.

[0021] In subsequent rounds of iterative sampling, the layer-by-layer collaborative deviation is calculated using the current round's sampling result and the collaborative constraint signal. The layer-by-layer collaborative deviation is then propagated in reverse to the sampling constraint potential to perform asymptotic constraint compaction, driving the next round of iterative sampling. The above deviation propagation and constraint compaction are repeated until the convergence condition is met. The reciprocal of the collaborative deviation of each round's sampling result is used as the weight to perform weighted aggregation, thus forming the target image.

[0022] The target image is segmented by salient regions, and the color features and edge contour features of each salient region are extracted to form a zonal visual feature representation. The surface three-dimensional contour point cloud data and material response parameters of the target medium are collected according to the target medium type identifier, including:

[0023] A visually salient gradient flow field is established for the target image. The gradient flow field induces the boundary energy field of the region. The boundary energy field drives the coordinated iteration of region growth and boundary contraction until the boundary energy converges. The final set of salient regions and the boundary energy intensity of each region are then determined.

[0024] For each salient region, a second-order correlation tensor of regional color is constructed. The energy-guided main color extraction is performed by combining the tensor feature decomposition results with the energy intensity of the region boundary. The main color vector and residual distribution are combined to form color features. The edge contour features are formed by combining the statistical distribution of the tangent direction field of the region boundary point set with the direction consistency coefficient. The two types of features are weighted and aggregated according to the energy intensity of each region boundary and arranged in spatial order to form a regional visual feature representation.

[0025] Based on the target medium type identifier, the physical property categories of the medium are decomposed. The point cloud acquisition resolution and scanning path are determined according to each property category. Parallel geometric scanning of multiple property categories is performed on the surface of the target medium. The scanning results of each category are fused into surface three-dimensional contour point cloud data. Special excitation methods are designed for each physical property category. Attribute classification controlled excitation measurement is performed on the target medium. The response signals of each channel are decoupled and merged according to the property category to construct material response parameters containing optical and permeability properties.

[0026] Based on the visual feature representation of the partitions and the point cloud data of the surface three-dimensional contour, the salient regions of the target image are projected and mapped onto the curvature field of the medium surface. An adaptive printhead motion trajectory oriented towards the curved medium is planned, and a partition-differentiated ink volume injection instruction set is generated in conjunction with the material response parameters, including:

[0027] A normal curvature flow field is established on the three-dimensional contour point cloud data of the surface. The surface curvature is divided into curvature partitions based on the streamline convergence distribution of the normal curvature flow field. The normal curvature principal direction vector and the representative value of streamline density of each curvature partition are extracted.

[0028] Based on the position of each salient region in the visual feature representation of the partition and the corresponding streamline density representative value of the curvature partition, the streamline density representative value is used as the deformation correction coefficient to perform flow field guidance unfolding projection along the normal curvature principal direction vector to obtain the projection mapping result;

[0029] Based on the projection mapping results and the principal direction vector of normal curvature of each curvature partition, the principal direction vector of normal curvature is used as the reference for printhead travel and normal direction. A continuous trajectory planning model for partition driven by normal curvature flow is established. The sub-trajectory of printhead motion is planned for each partition. The attitude of the transition segment is corrected by the angle between the principal direction vectors of normal curvature of adjacent partitions. The transition segments are then connected to form an adaptive printhead motion trajectory.

[0030] Based on the material response parameters, a streamline density-weighted ink volume attachment modulation function is established for each curvature zone. The ink volume reference and modulation coefficient are calculated, and the ink volume of each curvature zone is calculated by combining the streamline density representative value, thereby generating a zone-specific ink volume injection instruction set.

[0031] Based on the material response parameters, a streamline density-weighted ink volume attachment modulation function is established for each curvature zone. The ink volume baseline and modulation coefficient are calculated, and the ink volume for each curvature zone is calculated using the representative value of streamline density. This generates a zone-specific ink volume injection instruction set, including:

[0032] The material response parameters are subjected to adhesion-diffusion dual-component co-decomposition to extract the ink adhesion ability component and the ink lateral diffusion inhibition component on the medium surface. The adhesion ability component is used as the main modulation weight and the lateral diffusion inhibition component is used as the compensation correction term to construct the adhesion modulation initial function for each curvature zone.

[0033] The streamline density representative value of each curvature zone is mapped to an elastic weighted cumulative coefficient. The elastic weighted cumulative coefficient is used to apply a cross-zone density elastic cumulative gain to the adhesion modulation initial function to form a streamline density weighted ink amount adhesion modulation function for each curvature zone.

[0034] A spatial saliency weight field is constructed based on the color features of each salient region in the projection mapping result. An adaptive ink volume reference for each salient region is established using the spatial saliency weight field. A directional edge enhancement modulation is applied to the adaptive ink volume reference based on the edge contour feature density of each salient region. The modulated ink volume is substituted into the streamline density weighted ink volume attachment modulation function of each curvature partition to generate a partition-differentiated ink volume injection instruction set.

[0035] A second aspect of the present invention provides a cultural and creative product production system based on AI image generation and multi-media printing, comprising:

[0036] The input acquisition unit is used to acquire the topic semantic text description and reference style sketch input by the user, convert the topic semantic text description into a semantic feature vector through a text encoder, and extract the style feature vector from the reference style sketch through a visual encoder.

[0037] A cross-modal fusion unit is used to perform cross-modal attention fusion on the semantic feature vector and the style feature vector to generate a cross-modal joint guidance vector. The cross-modal joint guidance vector is used to apply joint constraints to the image generation model to generate a target image controlled by text semantics and visual style.

[0038] The feature extraction unit is used to perform salient region segmentation on the target image, extract the color features and edge contour features of each salient region to form a partitioned visual feature representation, and collect the surface three-dimensional contour point cloud data and material response parameters of the target medium according to the target medium type identifier.

[0039] The trajectory planning unit is used to project and map the salient region of the target image onto the curvature field of the medium surface based on the visual feature representation of the partition and the three-dimensional contour point cloud data of the surface, plan the adaptive printhead motion trajectory facing the curved medium, and generate a partition-differentiated ink volume injection instruction set in combination with the material response parameters.

[0040] The printing execution unit is used to perform surface adaptive printing output on the target medium according to the partitioned differentiated ink volume injection instruction set, so as to complete the production of cultural and creative products.

[0041] A third aspect of the present invention provides an electronic device, comprising:

[0042] processor;

[0043] Memory used to store processor-executable instructions;

[0044] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0045] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0046] This method enables intelligent, high-fidelity production of cultural and creative products from initial conception to finished product. Through multimodal guidance integrating textual semantics and visual sketches, the generated target image accurately expresses the user's intent while maintaining a personalized style, significantly enhancing the controllability and uniqueness of the creative design. Based on salient region segmentation and partitioned visual feature extraction, it achieves refined analysis of key image content. Combined with the three-dimensional contour and material parameters of the target medium, it intelligently maps planar images to complex curved surfaces, ensuring the morphological adaptability and visual integrity of the visual content on non-planar carriers. The curved surface projection mapping algorithm dynamically adjusts the spatial distribution of image regions according to the curvature of the medium, avoiding the deformation and distortion caused by traditional planar printing on curved media.

[0047] Adaptive print trajectory planning and zoned ink volume control technology enable high-precision curved surface printing output. The printhead movement trajectory is optimized in real time based on the geometric features of the media surface, ensuring printing accuracy and media adhesion. Zoned differentiated ink volume injection commands combine material absorption characteristics and regional color characteristics, achieving accurate color reproduction and clear presentation of details on various media, significantly improving the visual quality and texture of the finished product.

[0048] This method streamlines the technological process from digital creativity to physical form, lowering the barrier to entry for producing cultural and creative products on complex curved surfaces. Users only need to provide text and sketches to obtain customized, high-quality finished products, supporting diverse media and personalized designs, providing the cultural and creative industry with an efficient, flexible, and high-quality intelligent production solution. Attached Figure Description

[0049] Figure 1 This is a flowchart illustrating the cultural and creative product production method based on AI image generation and multi-media printing according to an embodiment of the present invention.

[0050] Figure 2 This is a flowchart illustrating a method for generating target images that are co-controlled by textual semantics and visual style, as described in an embodiment of the present invention. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0053] Figure 1 This is a flowchart illustrating the cultural and creative product production method based on AI image generation and multi-media printing according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0054] Obtain the user-input topic semantic text description and reference style sketch, convert the topic semantic text description into a semantic feature vector through a text encoder, and extract the style feature vector from the reference style sketch through a visual encoder;

[0055] The semantic feature vector and the style feature vector are fused across modal attention to generate a cross-modal joint guidance vector. The cross-modal joint guidance vector is used to apply joint constraints to the image generation model to generate a target image controlled by text semantics and visual style.

[0056] The target image is segmented into salient regions, and the color features and edge contour features of each salient region are extracted to form a partitioned visual feature representation. The surface three-dimensional contour point cloud data and material response parameters of the target medium are collected according to the target medium type identifier.

[0057] Based on the visual feature representation of the partition and the point cloud data of the surface three-dimensional contour, the salient region of the target image is projected and mapped onto the curvature field of the medium surface, an adaptive printhead motion trajectory for the curved medium is planned, and a partition-differentiated ink volume injection instruction set is generated in combination with the material response parameters.

[0058] Based on the partitioned differentiated ink volume injection instruction set, the target medium is subjected to surface adaptive printing output to complete the production of cultural and creative products.

[0059] The topic semantic text description is converted into a semantic feature vector using a text encoder, and the reference style sketch is used to extract a style feature vector using a visual encoder, including:

[0060] Semantic role labeling is performed on the topic semantic text description, the text content is decomposed into topic core word groups and style modification word groups, the topic core word groups and the style modification word groups are independently semantically modeled through the dual-path encoding sub-network of the text encoder, the fusion weights are assigned according to the semantic contribution of each encoding vector and weighted aggregation is performed, and the semantic feature vector that integrates topic and style information is output.

[0061] Superpixel segmentation is performed on the reference style sketch to obtain several semantically coherent superpixel regions. The pixel features in each superpixel region are averaged and pooled to obtain superpixel-level style descriptors. A region adjacency graph is constructed based on the spatial adjacency relationship between superpixels. The superpixel-level style descriptors are used as node features and input into the graph convolution module in the visual encoder. The graph convolution module propagates and aggregates the style information of adjacent regions layer by layer along the adjacent edges. The global readout layer of the visual encoder gathers the node features and outputs the style feature vector of the perceived region structure.

[0062] The user-provided semantic text description is fed into the semantic role labeling module for structured parsing. This module, based on dependency parsing algorithms, identifies verb predicates, noun entities, and adjective modifiers in the text, parsing the description "landscape imagery in the style of blue and white porcelain" into the core thematic word group "landscape, artistic conception" and the style modifier word group "blue and white porcelain, elegance, blank space". The two types of word groups are then input into a dual-path encoding sub-network within the text encoder. This sub-network employs an independent encoding layer based on the Transformer architecture, and the core thematic word group is extracted through an encoding path focused on semantic entity relationships. Dimensional features, style-modifying word groups are extracted through encoding paths that enhance emotional connotations and artistic attributes. Dimensional features.

[0063] After encoding, the semantic contribution of each feature is calculated. Weights are obtained by evaluating the syntactic importance of the core topic word groups in the description using an attention mechanism. The semantic coverage of style modifier word groups is weighted. Both satisfy the normalization constraint, and the fused semantic feature vector The vector, obtained by weighted aggregation, contains both topic entity information and style preference descriptions, providing semantic anchors for subsequent cross-modal fusion.

[0064] After the reference style sketch is input, the image is decomposed into several semantically coherent superpixel regions by a superpixel segmentation algorithm. A simple linear iterative clustering algorithm is used to iteratively cluster pixels based on their color space distance and coordinate space distance. Regions with continuous brushstrokes and similar textures in the sketch are merged into a single superpixel. Taking a sketch with an ink wash effect as an example, the areas with dark ink are aggregated into independent superpixels, while the blank areas form another semantic block. Mean pooling is performed on all pixel features in each superpixel region to compress high-dimensional pixel-level features into a fixed-dimensional superpixel-level style descriptor. This descriptor retains the average color distribution and texture roughness information within the region.

[0065] Based on the superpixel segmentation results, the system constructs a region adjacency graph to describe the spatial topological relationships between superpixels. If two superpixels share a common boundary on the image plane, a connection edge is established between the corresponding nodes in the adjacency graph. This adjacency graph uses superpixel-level style descriptors as node features, and the edge weights are set according to the visual similarity between adjacent regions. The adjacency graph is input into the graph convolution module within the visual encoder. This module achieves feature propagation through multiple convolution operations. During the l-th convolution, the node update rule is as follows: the current node feature and the features of all adjacent nodes are aggregated after being weighted by edge weights, and then transformed by a nonlinear activation function. After three layers of graph convolution propagation, the style information of the dark ink region gradually diffuses to the adjacent white space region, forming a node feature representation that perceives the global style distribution.

[0066] The global readout layer of the visual encoder employs graph attention pooling, calculating importance scores for all node features and assigning aggregation weights based on the contribution of each superpixel region to the overall style expression. For example, the ink-wash wash area, which dominates the main image in the sketch, receives a higher weight, while decorative textures at the edges receive relatively lower weights. The weighted aggregation outputs a style feature vector. Including information about regional structural relationships, this vector more accurately preserves the spatial logic and artistic composition features of brushstroke distribution in style sketches compared to traditional global average pooling methods, providing refined visual constraint signals for subsequent cross-modal attention fusion.

[0067] Figure 2 This is a flowchart illustrating a method for generating a target image controlled by the collaborative interaction of textual semantics and visual style, as described in an embodiment of the present invention. The method involves performing cross-modal attention fusion on the semantic feature vector and the style feature vector to generate a cross-modal joint guidance vector. This cross-modal joint guidance vector is then used to apply joint constraints to an image generation model, thereby generating a target image controlled by the collaborative interaction of textual semantics and visual style. The method includes:

[0068] Several semantic prototype vectors are extracted from the semantic feature vectors, and each semantic prototype vector represents the main distribution direction of the semantic feature vectors in a semantic subspace;

[0069] Using the semantic prototype vector as an anchor point, calculate the cross-modal response value between the style feature vector and each semantic prototype vector, perform selective activation on the style feature vector based on the cross-modal response value, and output the style activation vector anchored in the semantic subspace;

[0070] The style activation vector and the semantic feature vector are subjected to bidirectional cross-residual fusion. During the fusion process, the topic-dominant component of the semantic feature vector and the style-dominant component of the style activation vector are preserved. The fusion result is projected to a unified representation space through a feature mapping layer to output a cross-modal joint guiding vector.

[0071] The cross-modal joint guiding vector is decomposed into semantic constraint components and style constraint components. The semantic constraint components are injected into the semantic control path of the image generation model, and the style constraint components are injected into the style control path of the image generation model. The semantic control path and the style control path are fused collaboratively at the feature integration nodes of each layer. The image generation model performs multiple rounds of iterative sampling under the dual-path collaborative constraint, and the sampling results are collected to form the target image.

[0072] When performing cross-modal attention fusion, the semantic feature vectors are decomposed into prototypes. A clustering algorithm is used to divide the semantic feature vectors into regions in a high-dimensional semantic space. Each cluster center constitutes a semantic prototype vector. The number of semantic prototype vectors is dynamically determined according to the complexity of the topic semantic text description, and is usually set to 3 to 8. Each semantic prototype vector represents the main direction of the distribution of semantic feature vectors in a specific semantic subspace. For example, when the topic semantic text is described as "mountain and water scenery", semantic prototype vectors such as "mountain morphology", "water flow", and "vegetation distribution" can be extracted.

[0073] After obtaining the semantic prototype vector, it is used as an anchor point to calculate the cross-modal response value between the style feature vector and the style feature vector. Specifically, the cosine similarity between the style feature vector and each semantic prototype vector is calculated, and this similarity value is the cross-modal response value. The cross-modal response value reflects the response intensity of the style feature vector corresponding to each semantic prototype vector. Selective activation is performed on the style feature vector based on the cross-modal response value. A gating mechanism is used to map the cross-modal response value to the activation weight through the Sigmoid function, and then perform element-wise multiplication with the style feature vector to output the style activation vector anchored in the semantic subspace. This style activation vector retains the style components that strongly respond to the semantic prototype vector and suppresses style components that are not semantically related.

[0074] The style activation vector and semantic feature vector are fused using a bidirectional cross-residual fusion method to construct a semantic branch and a style branch. In the semantic branch, the style activation vector is mapped through a fully connected layer and then weighted and added to the semantic feature vector. The fusion weight is set to 0.3, preserving the topic-dominant component of the semantic feature vector. In the style branch, the semantic feature vector is mapped through a fully connected layer and then weighted and added to the style activation vector. The fusion weight is set to 0.7, preserving the style-dominant component of the style activation vector. The outputs of the two branches are concatenated and input into the feature mapping layer. The feature mapping layer consists of two fully connected network layers, with the intermediate layer having a dimension 1.5 times that of the fused feature vector. The activation function used is GELU. The output layer projects the fused features onto a unified representation space and outputs a cross-modal joint guiding vector.

[0075] The cross-modal joint guiding vector is decomposed into semantic constraint components and style constraint components using a learnable projection matrix. The semantic constraint components emphasize thematic content information, while the style constraint components emphasize artistic style information. The semantic constraint components are injected into the semantic control path of the image generation model, which controls the main content structure of the generated image. The style constraint components are injected into the style control path of the image generation model, which controls the color texture and artistic expression of the generated image. The semantic and style control paths are collaboratively fused at each residual block feature integration node of the image generation model, employing an adaptive feature modulation mechanism to dynamically adjust the fusion weights of the two paths based on the statistical characteristics of the current layer features. The image generation model performs 20 to 50 rounds of iterative sampling under dual-path collaborative constraints, progressively reconstructing image features that meet the constraints from random noise during each round. The iterative sampling process employs a denoising diffusion strategy, reducing noise levels and enhancing image details in each round. The intermediate results from each round of sampling are weighted and averaged, with the weights proportional to the number of sampling rounds. The final output target image simultaneously satisfies textual semantic accuracy and visual style consistency.

[0076] The semantic control pathway and style control pathway are collaboratively fused at each layer's feature integration node. The image generation model performs multiple rounds of iterative sampling under dual-path collaborative constraints, and the sampling results are aggregated to form the target image, including:

[0077] At each feature integration node, cross-path difference measurement is performed on the semantic layer features output by the semantic control path and the style layer features output by the style control path. The difference measurement result drives the difference compensation fusion, outputting the current layer integrated feature. The current layer integrated feature is superimposed on the subsequent layer input of the two paths in a residual manner. After each layer is completed in hierarchical order, the integrated features of each layer are weighted and aggregated according to the hierarchical weight to form a collaborative constraint signal.

[0078] The collaborative constraint signal is injected into the image generation model to form a dual-path collaborative constraint. In the first round of iterative sampling, the sampling constraint potential is initialized with the integrated features of each layer in the collaborative constraint signal to drive the first round of sampling and obtain the first round of sampling results.

[0079] In subsequent rounds of iterative sampling, the layer-by-layer collaborative deviation is calculated using the current round's sampling result and the collaborative constraint signal. The layer-by-layer collaborative deviation is then propagated in reverse to the sampling constraint potential to perform asymptotic constraint compaction, driving the next round of iterative sampling. The above deviation propagation and constraint compaction are repeated until the convergence condition is met. The reciprocal of the collaborative deviation of each round's sampling result is used as the weight to perform weighted aggregation, thus forming the target image.

[0080] During the image generation stage, the semantic control pathway and the style control pathway process the topic semantic feature vector and the visual style feature vector, respectively. The two pathways adopt a hierarchical processing architecture, with each pathway containing multiple feature transformation layers. The semantic control pathway focuses on maintaining the semantic integrity of the topic, and its layers gradually refine the semantic expression through convolution and normalization operations. The style control pathway focuses on capturing the brushstrokes, tones and compositional characteristics of the reference sketch, extracting texture statistical features through the Gram matrix and passing them through each layer.

[0081] At each feature integration node, extract the semantic layer features output from the i-th layer of the semantic control pathway. Style layer features output from the i-th layer of the style control pathway Perform cross-pathway difference measurement, specifically by calculating the cosine distance to measure the degree of deviation between the features of the two pathways:

[0082] ,

[0083] When d i A large value indicates a significant conflict between semantics and style at this layer. This difference measure is used to drive difference compensation fusion, constructing compensation weights:

[0084] ,

[0085] in Set the balance factor to 0.5 and perform weighted fusion. To obtain the integrated features of the current layer, The residual connection is superimposed on the input of the (i+1)th layer of the two paths to realize cross-layer information backflow. Each layer completes the above operation in the order from shallow to deep, forming a progressive feature integration chain.

[0086] After the integration chain is completed, the integration features of each layer are analyzed. According to hierarchical weight Weighted aggregation is performed, with the hierarchical weight increasing with layer depth. The weight for shallow layers is set to 0.1, and the weight for deep layers is set to 0.4. Linear interpolation is used for intermediate layers, and the weighted summation yields the collaborative constraint signal.

[0087] ,

[0088] This signal serves a dual purpose: semantic guidance and style constraint.

[0089] The cooperative constraint signal is injected into the diffusion model as a dual-path cooperative constraint. In the first round of sampling, the sampling constraint potential is initialized with the integrated features of each layer in the cooperative constraint signal. Specifically, C is mapped to the noise space to form the initial constraint field P0. Starting with random Gaussian noise z0, denoising sampling is performed under the guidance of the constraint field to obtain the first round of sampling result x1.

[0090] In the k-th iteration sampling, take the sampling result x of the current round. k Using the cooperative constraint signal C, calculate the layer-by-layer cooperative deviation, and then... k The signals are fed into the feature extraction network to obtain the responses of each layer. Integration features with corresponding layers Calculate the mean square error The deviation of each layer The process is then passed back to the sampled constraint potential, and the constraint potential is updated.

[0091] ,

[0092] in The update step size is set to 0.01. The updated constraint potential P k Imposing tighter constraints on sampling drives the (k+1)th iteration to generate x. k+1 .

[0093] Repeat the above deviation calculation, constraint potential update, and iterative sampling until the convergence condition is met. The convergence condition is set as the average change in cooperative deviation over three consecutive rounds being less than a threshold of 0.001. After convergence, perform weighted aggregation using the reciprocal of the cooperative deviation of each round's sampling results as the weight, and calculate the weight for the k-th round:

[0094] ,

[0095] in To calculate the mean deviation of each layer in the k-th round, perform a weighted summation:

[0096] ,

[0097] This constitutes the final target image, which simultaneously satisfies the dual constraints of textual semantic description and reference style sketch.

[0098] The target image is segmented by salient regions, and the color features and edge contour features of each salient region are extracted to form a zonal visual feature representation. The surface three-dimensional contour point cloud data and material response parameters of the target medium are collected according to the target medium type identifier, including:

[0099] A visually salient gradient flow field is established for the target image. The gradient flow field induces the boundary energy field of the region. The boundary energy field drives the coordinated iteration of region growth and boundary contraction until the boundary energy converges. The final set of salient regions and the boundary energy intensity of each region are determined. For each salient region, a second-order correlation tensor of the region color is constructed. The energy-guided main color extraction is performed by combining the tensor feature decomposition results with the boundary energy intensity of the region. The main color vector and the residual distribution are combined to form the color feature.

[0100] The edge contour features are jointly constructed by the statistical distribution of the tangent direction field of the regional boundary point set and the direction consistency coefficient. The two types of features are weighted and aggregated according to the energy intensity of each regional boundary, and arranged in spatial order to form a regional visual feature representation. The physical property categories of the medium are decomposed according to the target medium type identifier. The point cloud acquisition resolution and scanning path are determined according to each attribute category. Parallel geometric scanning of multiple attribute categories is performed on the surface of the target medium, and the scanning results of each category are fused into surface three-dimensional contour point cloud data.

[0101] Specialized excitation methods are designed for each physical property category. Attribute-classified controlled excitation measurements are performed on the target medium. The response signals of each channel are decoupled and merged according to the attribute category to construct material response parameters that include optical and permeability properties.

[0102] When establishing a visually salient gradient flow field for a target image, the gradient vector field of each feature channel is calculated by selecting multi-channel features such as brightness, color difference, and texture of the target image, and then weighted and fused to form a salient gradient flow field. In the analysis of cultural and creative artworks, such as the analysis of ancient blue and white porcelain images, the brightness gradient, blue-white difference gradient, and decorative texture gradient can be considered simultaneously to form a gradient flow field that can accurately indicate the trend of visual information changes on the porcelain surface.

[0103] Based on the generated gradient flow field, a boundary energy field is further induced. Specifically, the divergence calculation result of the gradient flow field is used as the basis for the boundary energy field, and Gaussian smoothing is used to optimize the energy field, making it exhibit high energy values ​​at the region boundaries and low energy values ​​in homogeneous regions. For image analysis of wood carvings, this energy field can accurately locate the intersection of the carving boundary and the material texture. A collaborative iterative process of region growth and boundary contraction is driven by the boundary energy field. Points with local saliency maxima are selected from the image as seed points, and region expansion begins based on the region growth criterion; simultaneously, the contraction rate of the boundary points is controlled according to the strength of the boundary energy field, achieving a dynamic balance between internal growth and boundary optimization. When analyzing ink paintings, this collaborative iteration can accurately distinguish between areas of varying ink density and brushstroke outlines, avoiding the blurred boundary problem that traditional segmentation methods struggle to handle.

[0104] When the boundary energy converges to a preset threshold or the number of iterations reaches an upper limit, the final set of salient regions and the boundary energy intensity of each region are determined. Each salient region is assigned a boundary energy intensity value, reflecting its importance in overall visual perception. For example, in analyzing brocade images, this process can identify different brocade pattern regions and distinguish the main pattern from the background pattern based on the boundary energy intensity. For each identified salient region, a second-order correlation tensor of the region's color is constructed. The color values ​​of all pixels within the region are organized into three-dimensional vectors, and the covariance matrix of these vectors is calculated to obtain the second-order correlation tensor describing the color distribution characteristics of the region. When analyzing ancient painted pottery images, this tensor can comprehensively capture the intrinsic correlation between various color channels in the painted pottery patterns, laying the foundation for subsequent color feature extraction.

[0105] Energy-guided primary color extraction is performed using tensor eigenvalue decomposition results combined with region boundary energy intensity. Eigenvalues ​​are decomposed into eigenvalues, arranged in descending order, and the corresponding eigenvectors indicate the direction of the primary color. The region boundary energy intensity is used as a weighting factor to extract the primary color vector. In analyzing traditional textile patterns, this method effectively extracts the primary and secondary hues of the fabric and distinguishes their visual importance. The primary color vector and residual distribution are combined to construct color features. The primary color vector represents the main hue of the region, while the residual distribution describes the deviation of colors within the region from the primary color; the combination forms a comprehensive color feature description. For example, in analyzing lacquerware, the primary color vector can extract the basic hue of the lacquerware, while the residual distribution can represent the fine texture and gloss variations of the lacquer surface.

[0106] Edge contour features are constructed using the statistical distribution of the tangent direction field of the region boundary point set and the direction consistency coefficient. The tangent direction of each boundary point is calculated along the region boundary to generate the boundary tangent direction field. Statistical analysis of the direction field yields a direction distribution histogram and direction consistency coefficient. In analyzing traditional paper-cutting artworks, this feature accurately characterizes the edge details and line characteristics of the paper-cutting work, distinguishing between smooth curves and sharp angles. Weighted aggregation of color features and edge contour features is performed based on the boundary energy intensity of each region. The boundary energy intensity of each salient region is used as a weighting factor to separately weight the color features and edge contour features of that region, and then aggregated to form a comprehensive regional feature representation. In the analysis of ceramic patterns, this weighting mechanism can highlight the more visually significant pattern features and weaken the influence of secondary background elements.

[0107] The comprehensive features of each region are arranged according to spatial order to form a zoning visual feature representation. Based on the spatial positional relationship of salient areas in the original image, the comprehensive features of the regions are arranged and organized to form a zoning visual feature representation that retains spatial relationship information. For example, when analyzing traditional murals, this representation can preserve the relative positional relationships of elements such as figures and scenery in the murals, facilitating subsequent overall style analysis.

[0108] Based on the target medium type, the physical properties of the medium are decomposed into categories such as reflectivity, absorptivity, scattering rate, and hardness, according to the medium type of the cultural and creative work (such as ceramics, wood, metal, textiles, etc.). When analyzing ancient bronzes, their physical properties can be decomposed into multiple categories such as surface reflection characteristics, metallic texture characteristics, and oxide layer characteristics, providing a basis for subsequent geometric scanning.

[0109] The point cloud acquisition resolution and scanning path are determined according to each attribute category. For different physical attribute categories, corresponding point cloud acquisition resolution and scanning path strategies are set. A high-density spiral scanning path is used for reflectivity-sensitive metallic surfaces, while a densely meshed sampling path is used for areas with rich texture details. When analyzing traditional lacquerware, high-resolution sampling can be used for glossy lacquer areas, while adaptive resolution scanning can be used for embossed pattern areas.

[0110] A multi-attribute parallel geometric scan is performed on the target medium surface. Utilizing a multi-channel sensor array, different physical properties of the target medium surface are scanned simultaneously to acquire surface geometric information. For example, when analyzing brocade artworks, information such as the geometric undulations of the fabric surface, the weaving structure, and the material properties of the threads can be acquired simultaneously. The scan results from each category are then fused into a 3D contour point cloud data of the surface. Through spatial registration and data fusion algorithms, the geometric scan results of different attribute categories are unified into the same coordinate system, forming a comprehensive 3D contour point cloud data. When analyzing traditional wood carvings, this fused point cloud can simultaneously represent the geometric shape of the wood carving, the direction of the wood grain, and the texture of the surface treatment.

[0111] Specialized excitation methods are designed for each physical property category. Based on the response characteristics of different physical properties, corresponding excitation methods are designed, including multi-angle light source illumination, electromagnetic wave irradiation of different wavelengths, and multi-frequency acoustic wave excitation. When analyzing silk, polarized light illumination with different incident angles can be used to capture the optical anisotropy characteristics of the silk surface. Controlled excitation measurements based on property classification are performed on the target medium. According to the designed specialized excitation methods, controlled excitation is applied to the target medium, and its response signals are collected. For example, when analyzing traditional pigments, different wavelengths of light can be applied to measure their spectral reflectance curves; for ancient paper artifacts, their hygroscopic response characteristics to different humidity environments can be measured.

[0112] The response signals from each channel are decoupled and merged according to attribute categories. The acquired multi-channel response signals are decoupled by attribute category, and the characteristic parameters of each attribute category are extracted and merged by category to form a material response parameter set. When analyzing traditional porcelain, optical reflection signals, surface scattering signals, and internal transmission signals can be decoupled separately to obtain parameters such as the glaze gloss characteristics and the light transmission characteristics of the body.

[0113] Material response parameters incorporating optical and permeability properties are constructed. Based on decoupled and converged response signals, a material response parameter model is established to describe the optical properties (such as reflectivity and scattering properties) and permeability properties (such as absorptivity and permeation depth) of the target medium. When analyzing traditional ink paintings, this parameter model can quantify the absorption characteristics of ink, the permeability of paper, and the ink-wash effect produced by their interaction, providing a precise reference for digital preservation and creation.

[0114] Based on the visual feature representation of the partitions and the point cloud data of the surface three-dimensional contour, the salient regions of the target image are projected and mapped onto the curvature field of the medium surface. An adaptive printhead motion trajectory oriented towards the curved medium is planned, and a partition-differentiated ink volume injection instruction set is generated in conjunction with the material response parameters, including:

[0115] A normal curvature flow field is established on the three-dimensional contour point cloud data of the surface. The surface curvature is divided into curvature partitions based on the streamline convergence distribution of the normal curvature flow field. The normal curvature principal direction vector and the streamline density representative value of each curvature partition are extracted. Based on the position of each significant region in the visual feature representation of the partition and the corresponding streamline density representative value of the curvature partition, the streamline density representative value is used as the deformation correction coefficient to perform flow field guided unfolding projection along the normal curvature principal direction vector to obtain the projection mapping result.

[0116] Based on the projection mapping results and the principal direction vector of normal curvature of each curvature partition, the principal direction vector of normal curvature is used as the reference for printhead travel and normal direction. A continuous trajectory planning model for partition driven by normal curvature flow is established. The sub-trajectory of printhead motion is planned for each partition. The attitude of the transition segment is corrected by the angle between the principal direction vectors of normal curvature of adjacent partitions. The transition segments are then connected to form an adaptive printhead motion trajectory.

[0117] Based on the material response parameters, a streamline density-weighted ink volume attachment modulation function is established for each curvature zone. The ink volume reference and modulation coefficient are calculated, and the ink volume of each curvature zone is calculated by combining the streamline density representative value, thereby generating a zone-specific ink volume injection instruction set.

[0118] After acquiring the 3D contour point cloud data of the target medium, the normal vector of each sampling point in the point cloud and the surface segment formed by its adjacent points are calculated. The principal curvature value at that point is obtained by fitting the second derivative tensor of the local surface. The normal curvature flow field is constructed using a streamline tracing algorithm along the direction of maximum principal curvature. Starting from the curvature extremum point, streamline families are generated according to the gradient descent direction. The streamline convergence region corresponds to the curvature concentration zone of the medium surface. The spatial density distribution of the streamline family is statistically analyzed. The streamline density abrupt boundary is used as the basis for curvature partitioning. The medium surface is decomposed into several sub-regions with similar curvature characteristics. Within each curvature partition, the unit vector of the principal curvature direction is selected as the principal normal curvature direction vector of that partition. This vector indicates the direction of the most severe surface curvature. The representative value of streamline density is obtained by calculating the number of streamlines per unit area within the partition. The larger the value, the more significant the surface curvature.

[0119] Based on the salient region segmentation results of the target image, the center coordinates and boundary range of each salient region in the planar image are determined. The spatial correspondence between the salient regions and the curvature partitions is matched, and the planar image regions are projected onto the corresponding curved surface partitions. The projection process adopts the flow field-guided unfolding method, using the streamline density representative value as the deformation correction coefficient to compensate for the area distortion generated during the surface unfolding. Specifically, along the normal curvature principal direction vector, the length ratio of the projected region is adjusted according to the correction coefficient so that the distribution of image content on the curved surface is consistent with the original design intent. The correction coefficient is positively correlated with the streamline density representative value. Regions with larger curvature require a larger unfolding compensation. After the projection is completed, the mapping position and deformation parameters of each salient region in the curved surface coordinate system are recorded to form the projection mapping result dataset.

[0120] The trajectory planning model establishes a local coordinate system based on the principal direction vector of the normal curvature of each curvature partition. The printhead moves along the principal direction within this coordinate system, with the nozzle normal always aligned with the surface normal. Within a single curvature partition, a grid-fill strategy is used to generate the printing path, with the path spacing determined according to the printing resolution requirements. At the boundary between adjacent curvature partitions, the angle between the principal direction vectors of the normal curvature of the two partitions is calculated. When the angle exceeds a set threshold, a transition segment trajectory is inserted. The transition segment uses a cubic spline curve to smoothly connect the two sub-trajectories. The curve control points are automatically adjusted according to the angle and the rate of curvature change to ensure continuous change in printhead attitude. The sub-trajectories of each partition and the transition segment are concatenated according to the printing sequence to form a continuous motion trajectory covering the entire media surface. The trajectory data includes the three-dimensional position coordinates of the printhead, the normal attitude angle, and the motion speed parameters.

[0121] Ink volume injection control needs to consider the combined effects of the ink absorption characteristics of the medium material and the surface geometry. Material response parameters include material porosity, surface roughness, and adsorption coefficient, which determine the baseline value of ink volume requirement per unit area. A streamline density-weighted ink volume adhesion modulation function establishes a mapping relationship between the representative value of streamline density and the ink volume correction amount. Areas with high curvature have an increased actual surface area per unit projected area due to surface extension, requiring increased ink injection to ensure color saturation. The modulation function adopts a piecewise linear or exponential form, and the modulation coefficient is obtained through experimental calibration. During calculation, the baseline ink volume value for each zone is first determined based on the material response parameters, then multiplied by the modulation coefficient corresponding to the representative value of streamline density to obtain the actual ink volume requirement for that zone. The ink volume values ​​are converted into printer control commands, including printhead switching timing, piezoelectric drive voltage amplitude, and pulse width parameters, forming a zone-differentiated ink volume injection command set synchronized with the trajectory data.

[0122] Based on the material response parameters, a streamline density-weighted ink volume attachment modulation function is established for each curvature zone. The ink volume baseline and modulation coefficient are calculated, and the ink volume for each curvature zone is calculated using the representative value of streamline density. This generates a zone-specific ink volume injection instruction set, including:

[0123] The material response parameters are subjected to adhesion-diffusion dual-component co-decomposition to extract the ink adhesion ability component and the ink lateral diffusion inhibition component on the medium surface. The adhesion ability component is used as the main modulation weight and the lateral diffusion inhibition component is used as the compensation correction term to construct the adhesion modulation initial function for each curvature zone.

[0124] The streamline density representative value of each curvature zone is mapped to an elastic weighted cumulative coefficient. The elastic weighted cumulative coefficient is used to apply a cross-zone density elastic cumulative gain to the adhesion modulation initial function to form a streamline density weighted ink amount adhesion modulation function for each curvature zone.

[0125] A spatial saliency weight field is constructed based on the color features of each salient region in the projection mapping result. An adaptive ink volume reference for each salient region is established using the spatial saliency weight field. A directional edge enhancement modulation is applied to the adaptive ink volume reference based on the edge contour feature density of each salient region. The modulated ink volume is substituted into the streamline density weighted ink volume attachment modulation function of each curvature partition to generate a partition-differentiated ink volume injection instruction set.

[0126] When performing a two-component co-decomposition of the material response parameters, the ink absorption and lateral penetration characteristics of the medium surface are modeled separately. The adhesion component characterizes the vertical adsorption strength of ink droplets on the medium surface, obtained by measuring the adsorption depth per unit amount of ink per unit time. The lateral diffusion inhibition component reflects the lateral diffusion resistance of ink droplets on the medium surface, obtained by measuring the diffusion radius after applying a quantitative amount of ink droplets under standard test conditions, and using the reciprocal of the diffusion radius as the inhibition component value. When constructing the initial function for adhesion modulation, the adhesion component is used as the main modulation weight w. a By directly multiplying by the ink volume baseline value, the lateral diffusion suppression component is converted into a compensation correction term. The initial function is formed by adding the results to the main modulation result in an additive manner. ,in This is the baseline for unmodulated ink volume.

[0127] The streamline density representative value for each curvature zone is obtained by statistically analyzing the spatial density of the printhead motion trajectory within that zone. Specifically, it is calculated as the ratio of the total trajectory length to the zone area. When mapping the streamline density representative value to an elastically weighted cumulative coefficient, a non-linear mapping relationship is used. A smaller cumulative coefficient is applied to areas with lower streamline density, while an increasing cumulative coefficient is applied to areas with higher streamline density, ensuring a more significant ink volume accumulation gain in high-density areas. The elastically weighted cumulative coefficient serves as a multiplicative gain factor. This is applied to the initial adhesion modulation function to form a streamline density-weighted ink volume adhesion modulation function. .

[0128] When constructing the spatial saliency weight field, the color saturation, brightness difference, and hue contrast of each salient region in the projection mapping result are weighted and fused to generate a two-dimensional spatially distributed saliency weight map. The position-adaptive ink volume reference is determined based on the weight value of each pixel in the saliency weight field. Regions with high saliency are assigned larger ink volume reference values ​​to enhance visual performance, while regions with low saliency use smaller reference values ​​to save ink. The edge contour feature density is obtained by calculating the cumulative sum of gradient magnitudes of edge pixels in each salient region. The more complex and dense the edge contour, the larger the cumulative sum of gradients. Directional edge enhancement modulation is achieved by applying an additional ink volume increment in the neighborhood of edge pixels. The increment magnitude is proportional to the gradient magnitude to ensure edge contour clarity. The modulated ink volume of each salient region is substituted into the streamline density-weighted ink volume attachment modulation function of the corresponding curvature partition to calculate the final ink volume injection value for that partition. The partitions are organized in order to form a partition-differentiated ink volume injection instruction set, which includes the printhead position coordinates, ink volume injection value, and jet timing control parameters for each partition.

[0129] This invention also provides a cultural and creative product production system based on AI image generation and multi-media printing, including:

[0130] The input acquisition unit is used to acquire the topic semantic text description and reference style sketch input by the user, convert the topic semantic text description into a semantic feature vector through a text encoder, and extract the style feature vector from the reference style sketch through a visual encoder.

[0131] A cross-modal fusion unit is used to perform cross-modal attention fusion on the semantic feature vector and the style feature vector to generate a cross-modal joint guidance vector. The cross-modal joint guidance vector is used to apply joint constraints to the image generation model to generate a target image controlled by text semantics and visual style.

[0132] The feature extraction unit is used to perform salient region segmentation on the target image, extract the color features and edge contour features of each salient region to form a partitioned visual feature representation, and collect the surface three-dimensional contour point cloud data and material response parameters of the target medium according to the target medium type identifier.

[0133] The trajectory planning unit is used to project and map the salient region of the target image onto the curvature field of the medium surface based on the visual feature representation of the partition and the three-dimensional contour point cloud data of the surface, plan the adaptive printhead motion trajectory facing the curved medium, and generate a partition-differentiated ink volume injection instruction set in combination with the material response parameters.

[0134] The printing execution unit is used to perform surface adaptive printing output on the target medium according to the partitioned differentiated ink volume injection instruction set, so as to complete the production of cultural and creative products.

[0135] A third aspect of the present invention provides an electronic device, comprising:

[0136] processor;

[0137] Memory used to store processor-executable instructions;

[0138] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0139] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0140] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0141] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for creating cultural and creative products based on AI image generation and multi-media printing, characterized in that, include: Obtain the user-input topic semantic text description and reference style sketch, convert the topic semantic text description into a semantic feature vector through a text encoder, and extract the style feature vector from the reference style sketch through a visual encoder; The semantic feature vector and the style feature vector are fused across modal attention to generate a cross-modal joint guidance vector. The cross-modal joint guidance vector is used to apply joint constraints to the image generation model to generate a target image controlled by text semantics and visual style. The target image is segmented into salient regions, and the color features and edge contour features of each salient region are extracted to form a partitioned visual feature representation. The surface three-dimensional contour point cloud data and material response parameters of the target medium are collected according to the target medium type identifier. Based on the visual feature representation of the partition and the point cloud data of the surface three-dimensional contour, the salient region of the target image is projected and mapped onto the curvature field of the medium surface, an adaptive printhead motion trajectory for the curved medium is planned, and a partition-differentiated ink volume injection instruction set is generated in combination with the material response parameters. Based on the partitioned differentiated ink volume injection instruction set, the target medium is subjected to surface adaptive printing output to complete the production of cultural and creative products.

2. The method according to claim 1, characterized in that, The topic semantic text description is converted into a semantic feature vector using a text encoder, and the reference style sketch is used to extract a style feature vector using a visual encoder, including: Semantic role labeling is performed on the topic semantic text description, the text content is decomposed into topic core word groups and style modification word groups, the topic core word groups and the style modification word groups are independently semantically modeled through the dual-path encoding sub-network of the text encoder, the fusion weights are assigned according to the semantic contribution of each encoding vector and weighted aggregation is performed, and the semantic feature vector that integrates topic and style information is output. Superpixel segmentation is performed on the reference style sketch to obtain several semantically coherent superpixel regions. The pixel features in each superpixel region are averaged and pooled to obtain superpixel-level style descriptors. A region adjacency graph is constructed based on the spatial adjacency relationship between superpixels. The superpixel-level style descriptors are used as node features and input into the graph convolution module in the visual encoder. The graph convolution module propagates and aggregates the style information of adjacent regions layer by layer along the adjacent edges. The global readout layer of the visual encoder gathers the node features and outputs the style feature vector of the perceived region structure.

3. The method according to claim 1, characterized in that, The semantic feature vector and the style feature vector are fused across modal attention to generate a cross-modal joint guidance vector. This cross-modal joint guidance vector is then used to apply joint constraints to the image generation model, generating a target image that is co-controlled by textual semantics and visual style. Several semantic prototype vectors are extracted from the semantic feature vectors, each of which represents the main distribution direction of the semantic feature vectors in a semantic subspace; the cross-modal response values ​​between the style feature vectors and each semantic prototype vector are calculated using the semantic prototype vectors as anchor points; selective activation is performed on the style feature vectors based on the cross-modal response values; and the style activation vector anchored in the semantic subspace is output. The style activation vector and the semantic feature vector are subjected to bidirectional cross-residual fusion. During the fusion process, the topic-dominant component of the semantic feature vector and the style-dominant component of the style activation vector are preserved. The fusion result is projected to a unified representation space through a feature mapping layer to output a cross-modal joint guiding vector. The cross-modal joint guiding vector is decomposed into semantic constraint components and style constraint components. The semantic constraint components are injected into the semantic control path of the image generation model, and the style constraint components are injected into the style control path of the image generation model. The semantic control path and the style control path are fused collaboratively at the feature integration nodes of each layer. The image generation model performs multiple rounds of iterative sampling under the dual-path collaborative constraint, and the sampling results are collected to form the target image.

4. The method according to claim 3, characterized in that, The semantic control pathway and style control pathway are collaboratively fused at each layer's feature integration node. The image generation model performs multiple rounds of iterative sampling under dual-path collaborative constraints, and the sampling results are aggregated to form the target image, including: At each feature integration node, cross-path difference measurement is performed on the semantic layer features output by the semantic control path and the style layer features output by the style control path. The difference measurement result drives the difference compensation fusion, outputting the current layer integrated feature. The current layer integrated feature is superimposed on the subsequent layer input of the two paths in a residual manner. After each layer is completed in hierarchical order, the integrated features of each layer are weighted and aggregated according to the hierarchical weight to form a collaborative constraint signal. The collaborative constraint signal is injected into the image generation model to form a dual-path collaborative constraint. In the first round of iterative sampling, the sampling constraint potential is initialized with the integrated features of each layer in the collaborative constraint signal to drive the first round of sampling and obtain the first round of sampling results. In subsequent rounds of iterative sampling, the layer-by-layer collaborative deviation is calculated using the current round's sampling result and the collaborative constraint signal. The layer-by-layer collaborative deviation is then propagated in reverse to the sampling constraint potential to perform asymptotic constraint compaction, driving the next round of iterative sampling. The above deviation propagation and constraint compaction are repeated until the convergence condition is met. The reciprocal of the collaborative deviation of each round's sampling result is used as the weight to perform weighted aggregation, thus forming the target image.

5. The method according to claim 1, characterized in that, The target image is segmented by salient regions, and the color features and edge contour features of each salient region are extracted to form a zonal visual feature representation. The surface three-dimensional contour point cloud data and material response parameters of the target medium are collected according to the target medium type identifier, including: A visually salient gradient flow field is established for the target image. The gradient flow field induces the boundary energy field of the region. The boundary energy field drives the coordinated iteration of region growth and boundary contraction until the boundary energy converges. The final set of salient regions and the boundary energy intensity of each region are then determined. For each salient region, a second-order correlation tensor of regional color is constructed. The energy-guided main color extraction is performed by combining the tensor feature decomposition results with the energy intensity of the region boundary. The main color vector and residual distribution are combined to form color features. The edge contour features are formed by combining the statistical distribution of the tangent direction field of the region boundary point set with the direction consistency coefficient. The two types of features are weighted and aggregated according to the energy intensity of each region boundary and arranged in spatial order to form a regional visual feature representation. Based on the target medium type identifier, the physical property categories of the medium are decomposed. The point cloud acquisition resolution and scanning path are determined according to each property category. Parallel geometric scanning of multiple property categories is performed on the surface of the target medium. The scanning results of each category are fused into surface three-dimensional contour point cloud data. Special excitation methods are designed for each physical property category. Attribute classification controlled excitation measurement is performed on the target medium. The response signals of each channel are decoupled and merged according to the property category to construct material response parameters containing optical and permeability properties.

6. The method according to claim 1, characterized in that, Based on the visual feature representation of the partitions and the point cloud data of the surface three-dimensional contour, the salient regions of the target image are projected and mapped onto the curvature field of the medium surface. An adaptive printhead motion trajectory oriented towards the curved medium is planned, and a partition-differentiated ink volume injection instruction set is generated in conjunction with the material response parameters, including: A normal curvature flow field is established on the three-dimensional contour point cloud data of the surface. The surface curvature is divided into curvature partitions based on the streamline convergence distribution of the normal curvature flow field. The normal curvature principal direction vector and the representative value of streamline density of each curvature partition are extracted. Based on the position of each salient region in the visual feature representation of the partition and the corresponding streamline density representative value of the curvature partition, the streamline density representative value is used as the deformation correction coefficient to perform flow field guidance unfolding projection along the normal curvature principal direction vector to obtain the projection mapping result; Based on the projection mapping results and the principal direction vector of normal curvature of each curvature partition, the principal direction vector of normal curvature is used as the reference for printhead travel and normal direction. A continuous trajectory planning model for partition driven by normal curvature flow is established. The sub-trajectory of printhead motion is planned for each partition. The attitude of the transition segment is corrected by the angle between the principal direction vectors of normal curvature of adjacent partitions. The transition segments are then connected to form an adaptive printhead motion trajectory. Based on the material response parameters, a streamline density-weighted ink volume attachment modulation function is established for each curvature zone. The ink volume reference and modulation coefficient are calculated, and the ink volume of each curvature zone is calculated by combining the streamline density representative value, thereby generating a zone-specific ink volume injection instruction set.

7. The method according to claim 1, characterized in that, Based on the material response parameters, a streamline density-weighted ink volume attachment modulation function is established for each curvature zone. The ink volume baseline and modulation coefficient are calculated, and the ink volume for each curvature zone is calculated using the representative value of streamline density. This generates a zone-specific ink volume injection instruction set, including: The material response parameters are subjected to adhesion-diffusion dual-component co-decomposition to extract the ink adhesion ability component and the ink lateral diffusion inhibition component on the medium surface. The adhesion ability component is used as the main modulation weight and the lateral diffusion inhibition component is used as the compensation correction term to construct the adhesion modulation initial function for each curvature zone. The streamline density representative value of each curvature zone is mapped to an elastic weighted cumulative coefficient. The elastic weighted cumulative coefficient is used to apply a cross-zone density elastic cumulative gain to the adhesion modulation initial function to form a streamline density weighted ink amount adhesion modulation function for each curvature zone. A spatial saliency weight field is constructed based on the color features of each salient region in the projection mapping result. An adaptive ink volume reference for each salient region is established using the spatial saliency weight field. A directional edge enhancement modulation is applied to the adaptive ink volume reference based on the edge contour feature density of each salient region. The modulated ink volume is substituted into the streamline density weighted ink volume attachment modulation function of each curvature partition to generate a partition-differentiated ink volume injection instruction set.

8. A cultural and creative production system based on AI image generation and multi-media printing, used to implement the method as described in any one of claims 1-7, characterized in that, include: The input acquisition unit is used to acquire the topic semantic text description and reference style sketch input by the user, convert the topic semantic text description into a semantic feature vector through a text encoder, and extract the style feature vector from the reference style sketch through a visual encoder. A cross-modal fusion unit is used to perform cross-modal attention fusion on the semantic feature vector and the style feature vector to generate a cross-modal joint guidance vector. The cross-modal joint guidance vector is used to apply joint constraints to the image generation model to generate a target image controlled by text semantics and visual style. The feature extraction unit is used to perform salient region segmentation on the target image, extract the color features and edge contour features of each salient region to form a partitioned visual feature representation, and collect the surface three-dimensional contour point cloud data and material response parameters of the target medium according to the target medium type identifier. The trajectory planning unit is used to project and map the salient region of the target image onto the curvature field of the medium surface based on the visual feature representation of the partition and the three-dimensional contour point cloud data of the surface, plan the adaptive printhead motion trajectory facing the curved medium, and generate a partition-differentiated ink volume injection instruction set in combination with the material response parameters. The printing execution unit is used to perform surface adaptive printing output on the target medium according to the partitioned differentiated ink volume injection instruction set, so as to complete the production of cultural and creative products.

9. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 7.

Images