Data processing method based on digital cultural creativity

Abstract

The application relates to the field of digital culture technology, in particular to a data processing method based on digital culture creativity, which comprises the following steps: acquiring culture creativity information input by a user to extract a plurality of initial culture creativity keywords; generating a culture creativity element set based on the initial culture creativity keywords; performing culture correlation analysis on each culture creativity element in the culture creativity element set to determine a plurality of culture creativity element combinations and construct a corresponding culture creativity element relationship graph; generating a corresponding initial culture creativity image, performing image-text conversion on the initial culture creativity image to obtain key feature text; performing semantic matching processing on the key feature text and the initial culture creativity keywords to obtain a key matching degree, and determining whether to output the initial culture creativity image to the user based on the key matching degree; after the initial culture creativity image is output to the user, receiving feedback information of the user to perform image optimization on the initial culture creativity image to obtain a target culture creativity image. The application can improve the accuracy of culture creativity image generation.

Description

Technical Field

[0001] This invention relates to the field of digital cultural technology, and in particular to a data processing method based on digital cultural creativity. Background Technology

[0002] With the deep integration of digital technology and the cultural and creative industries, digital cultural creativity has become an important direction for the transformation and upgrading of the cultural industry. Leveraging next-generation information technologies such as artificial intelligence, big data, and multimodal perception, the digital generation of cultural and creative content is undergoing a paradigm shift from "manual design" to "intelligent generation." Automated digital creative content generation not only improves the efficiency of creative content production but also provides new technological pathways for cultural inheritance and innovation.

[0003] In the field of digital cultural and creative image generation, existing technologies have made a series of beneficial explorations. A typical existing solution is the AIGC model-based method for generating cultural and creative product images. This method uses the BERT model to perform semantic parsing on the input descriptive text to obtain cultural keywords. Then, based on these keywords, it searches a cultural symbol database for relevant image features, color features, and shape features to obtain matching visual elements. Finally, it uses multimodal alignment technology to align the text semantics with the image features to generate cultural and creative product images. While this solution achieves an automated generation process from text description to cultural and creative images, it still has the following shortcomings: First, this solution relies solely on single-modal text input information and fails to fully utilize multimodal creative information such as voice and image references that users may provide. Second, the processing of cultural keywords in this solution remains at the retrieval and matching level, without systematically modeling the semantic and cultural relationships between the retrieved cultural and creative elements. This results in insufficient logical relationships and internal connections between cultural elements in the generated images, leading to low accuracy in generating cultural and creative images. Summary of the Invention

[0004] To address this issue, the present invention provides a data processing method based on digital cultural creativity to overcome the problem of low accuracy in the generation of cultural and creative images in the prior art.

[0005] To achieve the above objectives, the present invention provides a data processing method based on digital cultural creativity, comprising:

[0006] Step S1: Obtain cultural and creative information input by the user, and perform multimodal fusion analysis on the cultural and creative information to extract several initial cultural and creative keywords. The cultural and creative information includes at least one of text description information, voice description information, and image reference information.

[0007] Step S2: Based on each of the initial cultural and creative keywords, retrieve the corresponding cultural and creative elements from the preset cultural and creative knowledge base to generate a set of cultural and creative elements, wherein the set of cultural and creative elements includes several cultural and creative elements;

[0008] Step S3: Perform cultural association analysis on each cultural and creative element in the set of cultural and creative elements to determine several combinations of cultural and creative elements and construct the corresponding relationship map of cultural and creative elements;

[0009] Step S4: Generate corresponding initial cultural and creative images based on the combinations of each cultural and creative element and the relationship graph of the cultural and creative elements, and perform image-text conversion on the initial cultural and creative images to obtain key feature text;

[0010] Step S5: Perform semantic matching processing on the key feature text and each of the initial cultural and creative keywords to obtain the key matching degree, and determine whether to output the initial cultural and creative image to the user based on the key matching degree.

[0011] Step S6: After outputting the initial cultural and creative image to the user, receive the user's feedback information to optimize the initial cultural and creative image to obtain the target cultural and creative image.

[0012] Furthermore, in step S1, several initial cultural and creative keywords are extracted, including:

[0013] Step S11: Perform multimodal fusion analysis on the cultural and creative information to obtain the first cultural and creative feature, wherein the cultural and creative feature includes text features, speech features and image features;

[0014] Step S12: Determine the cultural and creative text based on the feature richness and feature complexity of the first cultural and creative feature, wherein the feature richness is determined based on the feature dimension of the first cultural and creative feature, and the feature complexity is determined based on the proportion of image features in the first cultural and creative feature.

[0015] Step S13: Perform entity recognition on the cultural and creative text to extract several initial cultural and creative keywords.

[0016] Further, in step S2, a set of cultural and creative elements is generated, including:

[0017] Step S21: Semantically expand each of the initial cultural and creative keywords to obtain several related keywords;

[0018] Step S22: Based on the matching degree between each of the relevant keywords and the element keywords corresponding to each cultural and creative element in the preset cultural and creative knowledge base, determine the cultural and creative elements corresponding to each initial cultural and creative keyword to generate a set of cultural and creative elements.

[0019] Furthermore, in step S3, several combinations of cultural and creative elements are determined, including:

[0020] Step S31: Obtain the cultural attribute metadata of each cultural and creative element in the cultural and creative element set, and perform cluster analysis on each cultural and creative element based on the cultural attribute metadata to obtain several cultural and creative cluster groups, wherein any cultural and creative cluster group includes several cultural and creative elements.

[0021] Step S32: Determine several combinations of cultural and creative elements based on the number of cultural and creative elements in each of the aforementioned cultural and creative clusters.

[0022] Further, in step S3, constructing the corresponding cultural and creative element relationship map includes:

[0023] Step S33: Generate several cultural and creative element nodes based on the core cultural and creative elements in each of the cultural and creative element combinations;

[0024] Step S34: Determine the connection relationship of each cultural and creative element node based on the correlation between each cultural and creative element and each cultural and creative element node in each combination of cultural and creative elements, so as to construct the corresponding cultural and creative element relationship map.

[0025] Further, in step S4, generating the corresponding initial cultural and creative image includes:

[0026] Step S41: Generate cultural and creative element constraints based on the cultural and creative element relationship graph to construct an image generation model;

[0027] Step S42: Input the combination of each of the cultural and creative elements into the image generation model to obtain the initial cultural and creative image output by the image generation model.

[0028] Further, in step S5, determining the key matching degree includes:

[0029] Step S51: Divide the key feature text into several key text segments and match them with each of the initial cultural and creative keywords to obtain the initial text segments corresponding to each initial cultural and creative keyword.

[0030] Step S52: Calculate the text matching degree between each of the initial cultural and creative keywords and the corresponding initial text fragments, and determine the key matching degree based on each text matching degree and text coverage.

[0031] Further, step S5 includes:

[0032] Based on the condition that the key matching degree is greater than the preset matching degree, the initial cultural and creative image is determined to be output to the user.

[0033] Furthermore, in step S6, the initial cultural and creative image is optimized, including:

[0034] Based on user feedback, determine whether the initial cultural and creative image meets the expected standards, and determine the image optimization method, including performing basic image optimization on the initial cultural and creative image, or redetermining the combination of cultural and creative elements to redetermine the initial cultural and creative image.

[0035] Further, step S6 includes:

[0036] Under the condition that the initial cultural and creative image meets the expected standards, basic image optimization is performed on the initial cultural and creative image;

[0037] If the initial cultural and creative image does not meet the expected standards, the combination of cultural and creative elements is re-determined in order to redetermine the initial cultural and creative image.

[0038] Compared with existing technologies, the beneficial effects of this invention lie in its ability to fully explore users' cultural and creative intentions from different dimensions through multimodal fusion analysis, thereby extracting initial cultural and creative keywords. Based on these keywords, cultural and creative elements are retrieved from a pre-set cultural and creative knowledge base, and a relationship graph of cultural and creative elements containing multi-level relationships is constructed. This improves the comprehensiveness and accuracy of cultural and creative information extraction and enhances the efficiency of cultural and creative element retrieval. Initial cultural and creative images are generated based on element combinations and relationship graphs. The generated images are verified for matching degree through image-text conversion and semantic matching to avoid deviation from user intentions and ensure the accuracy of cultural and creative image generation. Image optimization is performed based on user feedback to ensure that the final output target cultural and creative image accurately matches the user's personalized needs, further improving the accuracy of cultural and creative image generation.

[0039] Furthermore, this invention performs multimodal feature recognition on the cultural and creative information input by the user, simultaneously extracting text features, speech features, and image features. This enables the comprehensive capture of semantic information in the user's creative expression. By introducing two quantitative indicators, feature richness and feature complexity, the invention adaptively selects and determines the cultural and creative text for entity recognition from the multimodal features. This accurately reflects the core elements of the user's creativity and ensures the accuracy of the selection of cultural and creative elements.

[0040] Furthermore, this invention expands the semantics of each initial cultural and creative keyword to obtain several related keywords, overcoming the limitations and one-sidedness of the semantic expression of the user's original input keywords, fully exploring the potential semantic space of the keywords, and calculating the matching degree between each related keyword and the element keywords corresponding to each cultural and creative element in the preset cultural and creative knowledge base. This can accurately measure the semantic association strength between each related keyword and the candidate cultural and creative element, and improve the efficiency and accuracy of subsequent cultural and creative generation.

[0041] Furthermore, this invention acquires the cultural attribute metadata of each cultural and creative element, and uses this metadata to perform cluster analysis on the elements. This allows the selected elements to be grouped according to their inherent cultural key elements. By combining the number of elements in each cluster, several combinations of cultural and creative elements are determined, ensuring a balance between the cohesion of these combinations in cultural semantics and the reasonableness of their quantity, objectively reflecting the actual cultural correlation strength between the elements. Based on the core elements in each combination, several cultural and creative element nodes are generated, and the connection relationships between each element and each node in each combination are determined to construct a relationship graph. This highlights the expressive focus of each group of cultural and creative ideas, providing clear core creative constraints for image generation.

[0042] Furthermore, this invention divides key feature text into several key text segments and matches them with each initial cultural and creative keyword to obtain the initial text segment corresponding to each initial cultural and creative keyword. This achieves accurate correspondence and local positioning between the overall image description text and each creative keyword. By calculating the text matching degree between each initial cultural and creative keyword and its corresponding initial text segment, the accuracy of semantic expression of each creative keyword in its corresponding image description area can be objectively measured, thereby improving the accuracy of cultural and creative image generation. Attached Figure Description

[0043] Figure 1 This is a flowchart illustrating the data processing method based on digital cultural creativity according to an embodiment of the present invention;

[0044] Figure 2 This is a schematic diagram illustrating the process of extracting several initial cultural and creative keywords in an embodiment of the present invention;

[0045] Figure 3 This is a schematic diagram illustrating the process of generating a set of cultural and creative elements according to an embodiment of the present invention;

[0046] Figure 4 This is a schematic diagram illustrating the process of determining a combination of several cultural and creative elements in an embodiment of the present invention. Detailed Implementation

[0047] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0048] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0049] Please see Figure 1The diagram shown is a flowchart illustrating a data processing method based on digital cultural creativity according to an embodiment of the present invention; the method of this embodiment includes:

[0050] Step S1: Obtain cultural and creative information input by the user, and perform multimodal fusion analysis on the cultural and creative information to extract several initial cultural and creative keywords. The cultural and creative information includes at least one of text description information, voice description information, and image reference information.

[0051] In this embodiment, the cultural and creative information input by the user includes at least one of text description information, voice description information, and image reference information. For example, the user can input a descriptive text, such as "I want to design a silk scarf that incorporates Dunhuang flying apsara elements", a voice description, such as the user's dictated creative idea, or upload a reference image (such as a photo of a Dunhuang mural) through an input device (such as a keyboard, microphone, camera, or touch screen).

[0052] Please see Figure 2 The diagram illustrates the process of extracting several initial cultural and creative keywords according to an embodiment of the present invention. Specifically, in step S1, the extraction of several initial cultural and creative keywords includes:

[0053] Step S11: Perform multimodal fusion analysis on the cultural and creative information to obtain the first cultural and creative feature, wherein the cultural and creative feature includes text features, speech features and image features;

[0054] In this embodiment, the multimodal fusion parsing process includes: performing natural language preprocessing on the text description information, including word replacement, deduplication, and logical correction, to obtain the first text; performing speech recognition and text-to-speech processing on the speech description information, for example, using a deep learning-based automatic speech recognition model to convert speech into speech-text modal information, and performing natural language preprocessing, to obtain the second text; and performing image content recognition on the image reference information, for example, using a convolutional neural network-based object detection model to identify the main objects, scene layout, color distribution, and cultural symbols in the image, to fuse them into image-text modal information, and performing natural language preprocessing, to obtain the third text. Cross-modal semantic alignment and fusion of texts from different modalities are performed to obtain the first cultural and creative text. The first cultural and creative text includes a first text, a second text, and a third text. In practical applications, the first text, the second text, and the third text can be fused to perform structured and standardized descriptions. Feature extraction is performed on the first cultural and creative text, including word segmentation, part-of-speech tagging, entity recognition, etc. If the user only inputs one or two of the following: text description information, voice description information, and image reference information, for example, if the user inputs text description information and voice description information, then the corresponding text features and voice features are generated, while the image features are empty.

[0055] Step S12: Determine the cultural and creative text based on the feature richness and feature complexity of the first cultural and creative feature, wherein the feature richness is determined based on the feature dimension of the first cultural and creative feature, and the feature complexity is determined based on the proportion of image features in the first cultural and creative feature.

[0056] In this embodiment, the first cultural and creative feature includes dimensions such as texture, color, scene, object, and cultural symbol. Feature richness reflects the modal diversity covered by the cultural and creative information input by the user. The greater the feature richness, the richer the semantic capacity contained in the input information. Specifically, the feature dimensions of the first cultural and creative feature are counted and used as the feature richness. Feature complexity reflects the relative proportion of text modal information in the input information. The greater the feature complexity, the greater the difficulty in parsing the user input information. Specifically, the amount of text feature data, speech feature data, and image feature data in the first cultural and creative feature are counted, and the ratio of the amount of image feature data to the total amount of data is determined as the feature complexity. The total amount of data is the sum of the amount of text feature data, speech feature data, and image feature data.

[0057] Understandably, if the feature richness is greater than the preset richness and the feature complexity is less than the preset complexity, then the first cultural and creative text corresponding to the first cultural and creative feature is determined as the cultural and creative text. Otherwise, an information supplementation prompt is triggered to guide the user to supplement cultural and creative information, and multimodal fusion parsing is re-performed based on the supplemented cultural and creative information to update the first cultural and creative feature. The preset richness can be set based on the average feature dimension of a single input from a historical user, and the preset complexity can be set based on the average proportion of image features in the cultural and creative information input by a single user in a historical period.

[0058] Step S13: Perform entity recognition on the cultural and creative text to extract several initial cultural and creative keywords.

[0059] In this embodiment, the creative text is input into the pre-trained creative entity recognition model. The model has a built-in dictionary of intangible cultural heritage, national trend, and traditional aesthetics entities, which distinguishes between general modifiers, modal particles and creative professional entities. It identifies and extracts all cultural entity words, removes duplicate words, and outputs a number of initial creative keywords after weight sorting.

[0060] Specifically, the embodiments of the present invention perform multimodal feature recognition on the cultural and creative information input by users, and simultaneously extract text features, speech features and image features. This can comprehensively capture the semantic information in the user's creative expression. By introducing two quantitative indicators, feature richness and feature complexity, the invention adaptively selects and determines the cultural and creative text for entity recognition from the multimodal features. This can accurately reflect the core elements of the user's creativity and ensure the accuracy of the selection of cultural and creative elements.

[0061] Step S2: Based on each of the initial cultural and creative keywords, retrieve the corresponding cultural and creative elements from the preset cultural and creative knowledge base to generate a set of cultural and creative elements, wherein the set of cultural and creative elements includes several cultural and creative elements;

[0062] In this embodiment, the preset cultural and creative knowledge base is a pre-constructed structured database that stores several cultural and creative elements and their corresponding element keywords, cultural attribute metadata, and the relationships between elements. The cultural attribute metadata includes cultural origin, historical period, regional affiliation, symbolic meaning, style, and school of thought. The preset cultural and creative knowledge base can be constructed by digitally collecting, annotating, and structurally storing massive amounts of cultural resources, including but not limited to cultural relic data, folk customs materials, artworks, and historical documents.

[0063] Please see Figure 3 As shown, it is a schematic diagram of the process for generating a set of cultural and creative elements according to an embodiment of the present invention; specifically, in step S2, generating a set of cultural and creative elements includes:

[0064] Step S21: Semantically expand each of the initial cultural and creative keywords to obtain several related keywords;

[0065] In this embodiment, semantic expansion includes synonym expansion, hierarchical expansion, and association expansion. Synonym expansion can be based on word vector models (such as Word2Vec and GloVe) to find semantically similar words. For example, "flying apsara" can be expanded to "celestial musicians," "goddess of incense and music," and "flying goddess." Hierarchical expansion can be based on the cultural and creative knowledge graph for hierarchical expansion. For example, the hierarchical concept of "Dunhuang" is "Silk Road cultural heritage," and the hierarchical concepts are "Mogao Grottoes" and "Yulin Grottoes." The hierarchical concept of "tea set" is "vessel," and the hierarchical concepts are "teapot," "teacup," and "tea tray." Association expansion can be based on the association relationships in the knowledge graph. For example, "Dunhuang" is associated with "murals," "sutra illustrations," and "cave ceilings." "Flying apsara" is associated with "ribbons," "auspicious clouds," and "lotus seat." Through semantic expansion, each initial cultural and creative keyword can correspond to several related keywords. For example, the initial cultural and creative keyword "flying apsara" can be semantically expanded to obtain related keywords such as: celestial musicians, goddess of incense and music, ribbons, auspicious clouds, lotus seat, murals, and pipa playing in reverse.

[0066] Step S22: Based on the matching degree between each of the relevant keywords and the element keywords corresponding to each cultural and creative element in the preset cultural and creative knowledge base, determine the cultural and creative elements corresponding to each initial cultural and creative keyword to generate a set of cultural and creative elements.

[0067] In this embodiment, each cultural and creative element has several corresponding element keywords. If the matching degree between any related keyword and any element keyword of any cultural and creative element is greater than a preset threshold, then the cultural and creative element is determined to be the cultural and creative element corresponding to the related keyword. In practical applications, the matching degree can be calculated based on cosine similarity, which is existing technology and will not be elaborated further. The greater the matching degree between any related keyword and any element keyword of any cultural and creative element, the stronger its correlation. If there are related keywords and element keywords of multiple cultural and creative elements whose matching degrees are all greater than the preset threshold, then the cultural and creative element corresponding to the highest matching degree is determined to be the cultural and creative element corresponding to the initial cultural and creative keyword. In practical applications, since the value range of cosine similarity is -1 to 1, that is, the corresponding matching degree range is -1 to 1, if the preset threshold is set too high, it will lead to over-filtering, resulting in the subsequent generated image elements being monotonous and not meeting user needs. If the preset threshold is set too low, it will lead to too many cultural and creative elements that meet the conditions, increasing the difficulty of subsequent data processing, and the correlation between the elements will be small, resulting in the subsequent generated image elements being messy. The corresponding preset threshold can be adaptively set based on actual user needs.

[0068] Specifically, the embodiments of the present invention expand the semantics of each initial cultural and creative keyword to obtain several related keywords, thereby overcoming the limitations and one-sidedness of the semantic expression of the user's original input keywords, fully exploring the potential semantic space of the keywords, and calculating the matching degree between each related keyword and the element keywords corresponding to each cultural and creative element in the preset cultural and creative knowledge base. This can accurately measure the semantic association strength between each related keyword and the candidate cultural and creative element, thereby improving the efficiency and accuracy of subsequent cultural and creative generation.

[0069] Step S3: Perform cultural association analysis on each cultural and creative element in the set of cultural and creative elements to determine several combinations of cultural and creative elements and construct the corresponding relationship map of cultural and creative elements;

[0070] Please see Figure 4 As shown, this is a flowchart illustrating the process of determining a combination of several cultural and creative elements according to an embodiment of the present invention; specifically, in step S3, determining a combination of several cultural and creative elements includes:

[0071] Step S31: Obtain the cultural attribute metadata of each cultural and creative element in the cultural and creative element set, and perform cluster analysis on each cultural and creative element based on the cultural attribute metadata to obtain several cultural and creative cluster groups, wherein any cultural and creative cluster group includes several cultural and creative elements.

[0072] In this embodiment, the cultural attribute metadata of each cultural and creative element is converted into a feature vector. Specifically, for the attribute value of each dimension, one-hot encoding, embedding vector, or TF-IDF weighting are used to numerically represent it. The numerical results of all dimensions are concatenated to form the cultural attribute feature vector of the cultural and creative element. The cultural attribute feature vectors of each cultural and creative element are normalized to eliminate the difference in dimensions. Clustering algorithms are used to perform clustering operations on the normalized feature vectors. For example, K-Means clustering algorithm, hierarchical clustering algorithm, DBSCAN algorithm, or graph theory-based spectral clustering algorithm. Taking K-Means clustering algorithm as an example, the number of clusters K is preset by elbow rule or silhouette coefficient method, K cluster centers are randomly initialized, and each cultural and creative element is assigned to the cluster group corresponding to the cluster center most similar to its cultural attribute characteristics through iterative updates until the cluster center no longer changes. After clustering, several cultural and creative cluster groups are obtained, and each cultural and creative cluster group contains several cultural and creative elements with similar cultural attribute characteristics.

[0073] Step S32: Determine several combinations of cultural and creative elements based on the number of cultural and creative elements in each of the aforementioned cultural and creative clusters.

[0074] In this embodiment, for any cultural and creative cluster, if the number of cultural and creative elements in the cluster is greater than a preset number, then the cultural and creative elements in the cluster are combined into a corresponding cultural and creative element combination. The more cultural and creative elements in any cluster, the greater the importance of the cultural and creative connotation corresponding to the cluster to user needs. Implementers can set the preset number based on actual conditions or based on the sum of the mean and standard deviation of the number of cultural and creative elements in each cluster.

[0075] Specifically, in step S3, constructing the corresponding cultural and creative element relationship map includes:

[0076] Step S33: Generate several cultural and creative element nodes based on the core cultural and creative elements in each of the cultural and creative element combinations;

[0077] In this embodiment, the cluster center of the cultural and creative cluster group corresponding to the combination of cultural and creative elements is determined as the core cultural and creative element corresponding to the combination of cultural and creative elements, and the core cultural and creative element is determined as the cultural and creative element node.

[0078] Step S34: Determine the connection relationship of each cultural and creative element node based on the correlation between each cultural and creative element and each cultural and creative element node in each combination of cultural and creative elements, so as to construct the corresponding cultural and creative element relationship map.

[0079] In this embodiment, for any combination of cultural and creative elements, the average distance between its corresponding cultural and creative element node and other cultural and creative elements in the combination is determined as the corresponding correlation degree. The core cultural and creative element with the most cultural and creative elements in the combination is determined as the central cultural and creative element node, and the remaining cultural and creative element nodes are determined as auxiliary cultural and creative element nodes. Corresponding connection edges are constructed between the central cultural and creative element node and the auxiliary cultural and creative element nodes, and the ratio of the correlation degree of the auxiliary cultural and creative element node to the correlation degree of the central cultural and creative element node is determined as the corresponding connection edge weight, thereby constructing a relationship graph of cultural and creative elements.

[0080] Specifically, this invention acquires the cultural attribute metadata of each cultural and creative element, and then performs cluster analysis on these elements. This allows for the grouping of selected elements based on their inherent cultural key elements. By combining the number of elements in each cluster, several combinations of cultural and creative elements are determined, ensuring a balance between the cohesion of these combinations in cultural semantics and the reasonableness of their quantity, objectively reflecting the actual cultural correlation strength between the elements. Based on the core elements in each combination, several cultural and creative element nodes are generated, and the connection relationships between each element and each node in each combination are determined to construct a relationship graph. This highlights the expressive focus of each group of cultural and creative ideas, providing clear core creative constraints for image generation.

[0081] Step S4: Generate corresponding initial cultural and creative images based on the combinations of each cultural and creative element and the relationship graph of the cultural and creative elements, and perform image-text conversion on the initial cultural and creative images to obtain key feature text;

[0082] Specifically, in step S4, generating the corresponding initial cultural and creative image includes:

[0083] Step S41: Generate cultural and creative element constraints based on the cultural and creative element relationship graph to construct an image generation model;

[0084] In this embodiment, the constraints on cultural and creative elements include element existence constraints, primary and secondary element constraints, and style consistency constraints. Element existence constraints require that the central cultural and creative element node in the specified cultural and creative element relationship graph must exist in the generated image, and at least one of the auxiliary cultural and creative element nodes must exist. Primary and secondary element constraints require that the generated image is dominated by the central cultural and creative element node, which is the core expressive element of the image, while the auxiliary cultural and creative element nodes serve as auxiliary or embellishment expressive elements. Style consistency constraints require that all elements existing in the image share the same cultural attributes, such as having a common historical period, regional affiliation, or style school, to generate visual style unity constraint rules and ensure that a coordinated and consistent visual expression form is adopted when generating the image.

[0085] It is understandable that using cultural and creative element constraints as generation rules to construct an image generation model, in one specific implementation, the basic architecture of the image generation model is a generative adversarial network-based model, such as the StyleGAN series models, or a diffusion-based generative model, such as the Stable Diffusion model or the DALL-E series models, or an autoregressive generative model based on the Transformer architecture. The cultural and creative element constraints are transformed into conditional input parameters of the model; for example, element existence constraints and primary / secondary element constraints are transformed into attention weight allocation parameters in the model, and style consistency constraints are transformed into conditional embedding vectors in the model, thereby constructing an image generation model driven by the relationship graph of cultural and creative elements.

[0086] Step S42: Input the combination of each of the cultural and creative elements into the image generation model to obtain the initial cultural and creative image output by the image generation model.

[0087] In this embodiment, the cultural and creative elements in each combination of cultural and creative elements are used as conditional input tensors and input into the image generation model to perform inference operations. The image generation model samples and denoises in the latent space according to the element information and constraints contained in the conditional input tensors, and gradually synthesizes a visual image that matches the input conditions. During the generation process, the model automatically controls the area ratio and visual prominence of each element in the image according to the element existence constraints and the primary and secondary element constraints. According to the style consistency constraints, the model automatically arranges the style matching relationship and overlapping layers of each element. Finally, the image generation model outputs the initial cultural and creative image corresponding to the input combination of cultural and creative elements.

[0088] Understandably, the initial cultural and creative image is converted into text to obtain key feature text. In one specific implementation, an image description generation model is used, such as an encoder-decoder model based on the Transformer architecture, to process the initial cultural and creative image. Specifically, the initial cultural and creative image is input into the image description generation model. The model's image encoder extracts visual features from the image and generates a sequence of visual feature vectors. The model's text decoder generates corresponding natural language description text autoregressively based on the sequence of visual feature vectors. This description text details the visual elements, color distribution, composition layout, and overall style contained in the image, which is the key feature text.

[0089] Step S5: Perform semantic matching processing on the key feature text and each of the initial cultural and creative keywords to obtain the key matching degree, and determine whether to output the initial cultural and creative image to the user based on the key matching degree.

[0090] Specifically, in step S5, determining the key matching degree includes:

[0091] Step S51: Divide the key feature text into several key text segments and match them with each of the initial cultural and creative keywords to obtain the initial text segments corresponding to each initial cultural and creative keyword.

[0092] In this embodiment, the key feature text is segmented. Specifically, the key feature text is segmented into sentences using punctuation marks such as periods, question marks, and exclamation marks as delimiters, dividing the key feature text into several independent sentences or clauses. Keyword matching is then performed on each segmented key text fragment. For any initial cultural and creative keyword, a search is conducted within each text fragment. Using precise keyword matching or fuzzy matching based on semantic similarity, text fragments containing the initial cultural and creative keyword or its synonyms are found. If any key text fragment contains content matching the initial cultural and creative keyword, then that key text fragment is identified as the initial text fragment corresponding to the initial cultural and creative keyword. In a preferred embodiment, if an initial cultural and creative keyword is matched in multiple text fragments, the text fragment with the highest matching degree is selected as its corresponding initial text fragment. If no direct or synonymous match is found for any initial cultural and creative keyword in any text fragment, then the initial text fragment corresponding to that keyword is marked as empty, indicating that the keyword may not be fully expressed in the generated image.

[0093] Step S52: Calculate the text matching degree between each of the initial cultural and creative keywords and the corresponding initial text fragments, and determine the key matching degree based on each text matching degree and text coverage.

[0094] In this embodiment, the greater the text matching degree between any initial cultural and creative keyword and its corresponding initial text fragment, the more consistent the initial text fragment is with the initial cultural and creative keyword. Methods for calculating text matching degree include, but are not limited to, methods based on semantic vector similarity and string similarity based on edit distance, which are existing technologies and will not be elaborated further. Text coverage is determined based on the proportion of initial text fragments in the key feature text. A greater text coverage indicates a higher degree of overall semantic restoration of the user's creative intent by the initial cultural and creative image. Specifically, the ratio of the number of initial text fragments to the number of key text fragments can be determined as text coverage, and the product of the mean of each text matching degree and the text coverage degree can be determined as the key matching degree.

[0095] Specifically, the embodiments of the present invention divide key feature text into several key text segments and match them with each initial cultural and creative keyword to obtain the initial text segment corresponding to each initial cultural and creative keyword. This achieves accurate correspondence and local positioning between the overall image description text and each creative keyword. The text matching degree between each initial cultural and creative keyword and its corresponding initial text segment is calculated respectively, which can objectively measure the accuracy of the semantic expression of each creative keyword in its corresponding image description area and improve the accuracy of cultural and creative image generation.

[0096] Specifically, step S5 includes:

[0097] Based on the condition that the key matching degree is greater than the preset matching degree, the initial cultural and creative image is determined to be output to the user.

[0098] In this embodiment, the key matching degree is compared with the preset matching degree. If the key matching degree is greater than the preset matching degree, the initial cultural and creative image is determined to meet the matching requirements of the user's creative intent, and the initial cultural and creative image is identified as the image to be output and displayed to the user. If the key matching degree is less than or equal to the preset matching degree, the initial cultural and creative image is determined to not meet the matching requirements, the image is not output to the user, and the regeneration process is triggered. The preset matching degree can be set based on the actual situation or the average key matching degree calculated from historical image output records.

[0099] Step S6: After outputting the initial cultural and creative image to the user, receive the user's feedback information to optimize the initial cultural and creative image to obtain the target cultural and creative image.

[0100] In this embodiment, after outputting the initial cultural and creative image to the user, feedback information from the user regarding the initial cultural and creative image is received through a user interaction interface. This feedback information includes, but is not limited to, one or more of the following forms: the user's overall satisfaction rating of the image, for example, 1 to 5 points; selective annotation of specific elements in the image by the user, such as circling or marking areas of satisfaction or dissatisfaction; text comments or modification suggestions entered by the user; and the user's binary judgment (yes / no) regarding "whether it meets expectations." The received user feedback information is parsed, and based on the feedback content, it is determined whether the initial cultural and creative image meets the expected standards. The determination of the expected standards includes: if the user's satisfaction rating exceeds a preset rating threshold, for example, 4 points or above, or the user explicitly selects "meets expectations," or the user does not mark areas of dissatisfaction, or the user's text comments do not contain modification requests, then the initial cultural and creative image is determined to meet the expected standards; otherwise, it is determined not to meet the expected standards.

[0101] Specifically, in step S6, optimizing the initial cultural and creative image includes:

[0102] Based on user feedback, determine whether the initial cultural and creative image meets the expected standards, and determine the image optimization method, including performing basic image optimization on the initial cultural and creative image, or redetermining the combination of cultural and creative elements to redetermine the initial cultural and creative image.

[0103] Specifically, step S6 includes:

[0104] Under the condition that the initial cultural and creative image meets the expected standards, basic image optimization is performed on the initial cultural and creative image;

[0105] In this embodiment, under the condition that the initial cultural and creative image meets the expected standards, basic image optimization is performed on the initial cultural and creative image. Basic image optimization includes, but is not limited to: image resolution enhancement, for example, using a super-resolution reconstruction algorithm to increase the image resolution to the target resolution; image color correction, for example, automatically adjusting contrast, saturation, color temperature, etc., to make the image visually more comfortable; image denoising, for example, using filtering algorithms or deep learning denoising models to remove noise or artifacts that may exist in the generated image; image sharpening to enhance the image edge clarity; image size cropping and composition fine-tuning, adaptively cropping according to the preset output size to ensure that the main elements are centered or conform to the golden ratio composition. After basic image optimization processing, the system outputs the optimized image as the target cultural and creative image to the user.

[0106] If the initial cultural and creative image does not meet the expected standards, the combination of cultural and creative elements is re-determined in order to redetermine the initial cultural and creative image.

[0107] In this embodiment, if the initial cultural and creative image does not meet the expected standards, a regeneration process is triggered to redetermine the combination of cultural and creative elements in order to redetermine the initial cultural and creative image. Specifically, based on user feedback, the source of the problem is located, and one or more of the following adjustment operations are performed:

[0108] If a user points out that a specific element does not meet expectations, reduce the weight of that element in the current combination, and adjust the search or selection of alternative cultural and creative elements based on the preferences mentioned in the user feedback to form an updated combination of cultural and creative elements.

[0109] If a user points out that the overall composition or style does not meet expectations, adjust the constraint parameters in the image generation model. For example, adjust the attention weight allocation in the primary and secondary element constraints. Based on the adjusted constraints, rebuild or configure the image generation model, input the current combination of cultural and creative elements into the adjusted model again, and regenerate the initial cultural and creative image.

[0110] When users provide explicit modifications to the text description, the modified description is integrated with the original cultural and creative information, and multimodal fusion parsing is re-executed to update the initial cultural and creative keywords, thereby regenerating a completely new initial cultural and creative image.

[0111] This invention utilizes multimodal fusion analysis to fully explore users' cultural and creative intentions from different dimensions, thereby extracting initial cultural and creative keywords. Based on these keywords, cultural and creative elements are retrieved from a pre-set cultural and creative knowledge base, and a multi-level relationship graph of these elements is constructed. This improves the comprehensiveness and accuracy of cultural and creative information extraction and enhances the efficiency of cultural and creative element retrieval. Initial cultural and creative images are generated based on element combinations and the relationship graph. Image-text conversion and semantic matching are used to verify the matching degree of the generated images, preventing them from deviating from user intentions and ensuring the accuracy of cultural and creative image generation. Image optimization is performed based on user feedback to ensure that the final output target cultural and creative image accurately matches the user's personalized needs, further improving the accuracy of cultural and creative image generation.

[0112] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A data processing method based on digital cultural creativity, characterized in that, include: Step S1: Obtain cultural and creative information input by the user, and perform multimodal fusion analysis on the cultural and creative information to extract several initial cultural and creative keywords. The cultural and creative information includes at least one of text description information, voice description information, and image reference information. Step S2: Based on each of the initial cultural and creative keywords, retrieve the corresponding cultural and creative elements from the preset cultural and creative knowledge base to generate a set of cultural and creative elements, wherein the set of cultural and creative elements includes several cultural and creative elements; Step S3: Perform cultural association analysis on each cultural and creative element in the set of cultural and creative elements to determine several combinations of cultural and creative elements and construct the corresponding relationship map of cultural and creative elements; Step S4: Generate corresponding initial cultural and creative images based on the combinations of each cultural and creative element and the relationship graph of the cultural and creative elements, and perform image-text conversion on the initial cultural and creative images to obtain key feature text; Step S5: Perform semantic matching processing on the key feature text and each of the initial cultural and creative keywords to obtain the key matching degree, and determine whether to output the initial cultural and creative image to the user based on the key matching degree. Step S6: After outputting the initial cultural and creative image to the user, receive the user's feedback information to optimize the initial cultural and creative image to obtain the target cultural and creative image.

2. The data processing method based on digital cultural creativity according to claim 1, characterized in that, In step S1, several initial cultural and creative keywords are extracted, including: Step S11: Perform multimodal fusion analysis on the cultural and creative information to obtain the first cultural and creative feature, wherein the cultural and creative feature includes text features, speech features and image features; Step S12: Determine the cultural and creative text based on the feature richness and feature complexity of the first cultural and creative feature, wherein the feature richness is determined based on the feature dimension of the first cultural and creative feature, and the feature complexity is determined based on the proportion of image features in the first cultural and creative feature. Step S13: Perform entity recognition on the cultural and creative text to extract several initial cultural and creative keywords.

3. The data processing method based on digital cultural creativity according to claim 1, characterized in that, In step S2, a set of cultural and creative elements is generated, including: Step S21: Semantically expand each of the initial cultural and creative keywords to obtain several related keywords; Step S22: Based on the matching degree between each of the relevant keywords and the element keywords corresponding to each cultural and creative element in the preset cultural and creative knowledge base, determine the cultural and creative elements corresponding to each initial cultural and creative keyword to generate a set of cultural and creative elements.

4. The data processing method based on digital cultural creativity according to claim 1, characterized in that, In step S3, several combinations of cultural and creative elements are determined, including: Step S31: Obtain the cultural attribute metadata of each cultural and creative element in the cultural and creative element set, and perform cluster analysis on each cultural and creative element based on the cultural attribute metadata to obtain several cultural and creative cluster groups, wherein any cultural and creative cluster group includes several cultural and creative elements. Step S32: Determine several combinations of cultural and creative elements based on the number of cultural and creative elements in each of the aforementioned cultural and creative clusters.

5. The data processing method based on digital cultural creativity according to claim 4, characterized in that, In step S3, the corresponding cultural and creative element relationship map is constructed, including: Step S33: Generate several cultural and creative element nodes based on the core cultural and creative elements in each of the cultural and creative element combinations; Step S34: Determine the connection relationship of each cultural and creative element node based on the correlation between each cultural and creative element and each cultural and creative element node in each combination of cultural and creative elements, so as to construct the corresponding cultural and creative element relationship map.

6. The data processing method based on digital cultural creativity according to claim 5, characterized in that, In step S4, the corresponding initial cultural and creative image is generated, including: Step S41: Generate cultural and creative element constraints based on the cultural and creative element relationship graph to construct an image generation model; Step S42: Input the combination of each of the cultural and creative elements into the image generation model to obtain the initial cultural and creative image output by the image generation model.

7. The data processing method based on digital cultural creativity according to claim 1, characterized in that, In step S5, determining the key matching degree includes: Step S51: Divide the key feature text into several key text segments and match them with each of the initial cultural and creative keywords to obtain the initial text segments corresponding to each initial cultural and creative keyword. Step S52: Calculate the text matching degree between each of the initial cultural and creative keywords and the corresponding initial text fragments, and determine the key matching degree based on each text matching degree and text coverage.

8. The data processing method based on digital cultural creativity according to claim 7, characterized in that, Step S5 includes: Based on the condition that the key matching degree is greater than the preset matching degree, the initial cultural and creative image is determined to be output to the user.

9. The data processing method based on digital cultural creativity according to claim 1, characterized in that, In step S6, the initial cultural and creative image is optimized, including: Based on user feedback, determine whether the initial cultural and creative image meets the expected standards, and determine the image optimization method, including performing basic image optimization on the initial cultural and creative image, or redetermining the combination of cultural and creative elements to redetermine the initial cultural and creative image.

10. The data processing method based on digital cultural creativity according to claim 9, characterized in that, Step S6 includes: Under the condition that the initial cultural and creative image meets the expected standards, basic image optimization is performed on the initial cultural and creative image; If the initial cultural and creative image does not meet the expected standards, the combination of cultural and creative elements is re-determined in order to redetermine the initial cultural and creative image.

Images