Art design feature conflict detection method and system based on deep learning
By using deep learning methods to detect conflicts in art and design features, and utilizing multimodal feature extraction networks and semantic mapping technology, the automated quantitative analysis of art and design features is achieved, improving detection efficiency and the objectivity of results, and solving the problems of low efficiency and strong subjectivity of traditional manual detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- GUANGXI MODERN VOCATIONAL & TECH COLLEGE
- Filing Date
- 2026-04-28
- Publication Date
- 2026-06-26
Smart Images
Figure CN122289738A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of design conflict detection, and in particular relates to a method and system for detecting art design feature conflicts based on deep learning. Background Technology
[0002] With the rapid development of digitalization and intelligentization in the field of art and design, computer vision technology based on deep learning has emerged in recent years. This technology can automatically extract multi-dimensional features through neural networks, achieve high-precision and high-efficiency image analysis and semantic understanding, and promote the automation of feature conflict detection in art and design.
[0003] In traditional techniques, designers primarily rely on visual observation and professional experience to subjectively compare and visually evaluate elements such as color, texture, and layout of target works. They identify potential conflicts through repeated adjustments and trial and error, such as identifying color styles or elements similar to existing artistic features. Current traditional manual inspection methods suffer from the following problems: low efficiency, relying on manual item-by-item checks, making it difficult to handle large-scale or complex design works; strong subjectivity, with different designers using different judgment standards, leading to inconsistent results and a lack of repeatability; and difficulty in quantifying and analyzing complex conflict relationships between design elements, such as multi-scale feature interactions and semantic hierarchical associations, thus limiting the scientific and refined development of art and design. Summary of the Invention
[0004] Therefore, it is necessary to provide a deep learning-based method and system for detecting art and design feature conflicts that can solve the above problems.
[0005] Firstly, this application provides a deep learning-based method for detecting art and design feature conflicts, including:
[0006] The image data of the target work is acquired, and based on the image data, a multimodal feature extraction network is used to extract multidimensional feature vectors.
[0007] Multi-scale decomposition is performed on the multi-dimensional feature vectors to obtain the multi-scale feature matrix.
[0008] The multi-scale feature matrix is mapped to a preset semantic space to obtain the semantic feature matrix.
[0009] Calculate the feature similarity between the semantic feature matrix and each benchmark feature matrix in the benchmark feature matrix library to obtain the feature similarity matrix.
[0010] Based on the feature similarity matrix, an art feature conflict detection report is generated.
[0011] In one embodiment, based on image data, a multimodal feature extraction network is used to extract multi-dimensional feature vectors, including:
[0012] The image data is input into the multimodal feature extraction network, and the following operations are performed:
[0013] Extract pixel space features from image data to obtain pixel space vectors;
[0014] Color information of an image is extracted based on pixel space vectors, and spatial quantization is performed on the color information to generate a color distribution statistical matrix.
[0015] Based on the color distribution statistical matrix, a semantic embedding encoder is used to perform feature semantic mapping to obtain a color semantic feature vector.
[0016] Perform instance segmentation on the image data, identify design element regions, and generate design element masks based on the design element regions;
[0017] By integrating pixel space vectors, color distribution statistics matrix, color semantic feature vectors, and design element masks, a multi-dimensional feature vector is obtained.
[0018] In one embodiment, the multi-dimensional feature vector is decomposed into a multi-scale feature matrix, including:
[0019] According to the preset scale, multi-level feature extraction is performed on the pixel space vector to generate a multi-level feature matrix, which includes a detail layer, a texture layer and a contour layer.
[0020] Based on the design element mask, spatial topological relationships are analyzed using morphological closing operations.
[0021] Based on the color distribution statistical matrix and spatial topological relationships, values are assigned to the detail layer, texture layer and contour layer to obtain a multi-scale feature matrix.
[0022] In one embodiment, the multi-scale feature matrix is mapped to a preset semantic space to obtain a semantic feature matrix, including:
[0023] Feature dimension clustering is performed on the feature matrices in the detail layer, texture layer and contour layer of the multi-scale feature matrix to generate hierarchical-scale feature sub-blocks;
[0024] Channel concatenation is performed on hierarchical-scale feature sub-blocks to generate a fused feature volume;
[0025] The fused feature volume is input into a pre-trained 3D convolutional neural network to extract multi-layer semantic feature maps;
[0026] Cross-scale correlation calculations are performed on the multi-layer semantic feature maps to obtain the semantic feature matrix.
[0027] In one embodiment, the baseline feature matrix library is constructed in a preset semantic space using labeled feature data of typical art and design works; the dimension of the baseline feature matrix is consistent with that of the semantic feature matrix.
[0028] Calculate the feature similarity between the semantic feature matrix and each benchmark feature matrix in the benchmark feature matrix library to obtain the feature similarity matrix, including:
[0029] Based on the semantic feature matrix and the benchmark feature matrix library, a set of positive and negative sample pairs is constructed using a triplet sampling strategy, wherein the anchor samples of the set of positive and negative sample pairs are selected from the semantic feature matrix of the target work.
[0030] The positive and negative sample pairs are aligned to obtain a cross-modal feature set;
[0031] The spatial similarity score of the cross-modal feature set is calculated using the contrastive loss function, and the feature similarity matrix is obtained.
[0032] In one embodiment, cross-scale correlation calculation is performed on the multi-layer semantic feature maps to obtain the semantic feature matrix, using the following formula:
[0033]
[0034] in, Here, M represents the semantic feature matrix, and M represents the number of multi-head attention heads. Let j be the dynamic weight coefficient of the j-th attention head. For the j-th query matrix, Let j be the value matrix of the attention head. Let be the learnable projection matrix of the j-th attention head. For a single attention head feature dimension, This is the channel attention adjustment coefficient. For channel attention function, For aggregated channel features, This is the Sigmoid activation function.
[0035] In one embodiment, an art feature conflict detection report is generated based on the feature similarity matrix, including:
[0036] A similarity threshold is set based on the feature similarity matrix, and regions whose similarity in each dimension exceeds the corresponding similarity threshold are extracted from the feature similarity matrix to obtain conflict feature regions;
[0037] Based on each conflict feature region, a feature conflict heatmap is generated;
[0038] Spatially aggregate the heatmaps of conflicting features to generate an aggregated conflict heatmap, and based on the aggregated conflict heatmap, divide the heatmap into graded heat regions according to a preset heat value grading rule.
[0039] Based on the spatial topological relationships of each thermal region and each design element mask, a spatial correlation network is established;
[0040] By integrating and aggregating conflict heatmaps, heat regions at various levels, and spatial correlation networks, an art feature conflict detection report is generated.
[0041] Secondly, this application also provides a deep learning-based art and design feature conflict detection system, including:
[0042] The image data acquisition module is used to acquire image data of the target work, and based on the image data, a multimodal feature extraction network is used to extract multi-dimensional feature vectors.
[0043] The multimodal feature extraction module is used to perform multi-scale decomposition on multi-dimensional feature vectors to obtain multi-scale feature matrices;
[0044] The multi-scale feature decomposition module maps the multi-scale feature matrix to a preset semantic space to obtain the semantic feature matrix;
[0045] The semantic space mapping module is used to calculate the feature similarity between the semantic feature matrix and each benchmark feature matrix in the benchmark feature matrix library, and obtain the feature similarity matrix.
[0046] The conflict report generation module is used to generate art feature conflict detection reports based on the feature similarity matrix.
[0047] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described deep learning-based art design feature conflict detection method.
[0048] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the above-described deep learning-based art design feature conflict detection method.
[0049] The aforementioned deep learning-based art and design feature conflict detection method and system acquires image data of the target artwork and uses a multimodal feature extraction network to extract multi-dimensional feature vectors, achieving comprehensive feature extraction of the artwork. It then performs multi-scale decomposition of the multi-dimensional feature vectors to generate a multi-scale feature matrix, enabling quantitative analysis of multi-level features such as details, textures, and contours of design elements. The multi-scale feature matrix is mapped to a preset semantic space to obtain a semantic feature matrix, enhancing the objectivity and consistency of feature understanding. The system calculates the feature similarity matrix between the semantic feature matrix and a benchmark feature matrix library, achieving accurate cross-modal feature comparison. Finally, it generates an art feature conflict detection report based on the feature similarity matrix, presenting the results in a visual format, improving detection efficiency and intuitiveness. Attached Figure Description
[0050] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0051] Figure 1 This is a flowchart of the deep learning-based art design feature conflict detection method of the present invention;
[0052] Figure 2 This is a structural diagram of the deep learning-based art design feature conflict detection system of the present invention. Detailed Implementation
[0053] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0054] In one embodiment, such as Figure 1As shown, a deep learning-based method for detecting feature conflicts in art and design is provided. This embodiment illustrates the application of this method to a design terminal. It is understood that this method can also be applied to a server, or to a system including both a terminal and a server, and is implemented through interaction between the terminal and the server. In this embodiment, the terminal can be a computer, intelligent design terminal, or tablet device with image acquisition, data processing, and visualization output capabilities. The server can be a cloud-based parallel server or GPU cluster server, configured with a storage module, a baseline feature matrix library storage unit, and a neural network computing module. The terminal and server establish a data interaction link through a communication network. In scenarios such as art and design, graphic design, and visual compliance review, when the design entity or review agency has needs for automated detection of feature conflicts in design works, similarity checks of design elements, and multi-dimensional design compliance verification, the terminal uploads the collected target work image data to the server. After completing the feature processing and conflict detection calculation process, the server generates a visual detection report and sends it back to the terminal for interactive output.
[0055] In this embodiment, the method includes the following steps:
[0056] S01: Obtain image data of the target artwork, and based on the image data, use a multimodal feature extraction network to extract multi-dimensional feature vectors.
[0057] Optionally, the image data of the target work can be digital image information containing artistic and design elements, such as single or multiple frame image data of various graphic designs, visual arts, and digital creative works. This data can be obtained through various methods such as image acquisition, scanning, and digital file import via a design terminal. The multimodal feature extraction network is a deep learning neural network model with multi-branch parallel feature extraction capabilities. It can extract design feature information from different modalities of the input image data. By extracting, mapping, and fusing features from multiple dimensions such as pixel space, color semantics, and design element entities, it generates a multi-dimensional feature vector that can characterize the design attributes of the target work. This multi-dimensional feature vector can cover the core design feature dimensions of the artistic and design work, providing a basis for conflict detection.
[0058] S02, perform multi-scale decomposition on the multi-dimensional feature vectors to obtain the multi-scale feature matrix.
[0059] Optionally, multi-scale decomposition refers to the multi-level and multi-resolution feature decomposition and reconstruction of feature vectors according to a preset feature level granularity, including various feature level splitting algorithms such as feature pyramid decomposition, hierarchical convolution extraction, and multi-resolution filtering. The design terminal can perform feature extraction, spatial topology analysis, and feature assignment processing on feature data of different dimensions in multi-dimensional feature vectors according to a preset scale level (such as from detail to global), generating a multi-scale feature matrix containing design feature information of different scale levels. This multi-scale feature matrix can completely represent the design feature attributes of the target work at different resolutions and levels.
[0060] S03, map the multi-scale feature matrix to the preset semantic space to obtain the semantic feature matrix.
[0061] Optionally, the preset semantic space refers to a high-dimensional feature space pre-constructed based on standardized semantic rules in the field of art and design and labeled feature data of typical art and design works. It has a unified feature dimension and semantic representation specification and can be used to achieve semantic alignment of design features of different scales and modalities. The design terminal can perform feature fusion, semantic encoding and spatial mapping on the multi-scale feature matrix through various feature semantic mapping processing methods such as pre-trained semantic coding network, feature embedding model, and cross-scale feature association algorithm, and uniformly map the underlying multi-scale features to the preset semantic space to generate a semantic feature matrix with dimension matching and design semantic representation capabilities.
[0062] S04. Calculate the feature similarity between the semantic feature matrix and each benchmark feature matrix in the benchmark feature matrix library to obtain the feature similarity matrix.
[0063] Optionally, the baseline feature matrix library is a standardized feature database pre-constructed within a preset semantic space, based on labeled feature data such as typical design works, standard design paradigms, and compliant design rules in the field of art and design. It includes multiple sets of baseline feature matrices that match the semantic feature matrix across various dimensions. The design terminal can employ algorithms for feature similarity quantification, such as feature distance measurement, similarity encoding calculation, feature matching comparison, and loss function calculation, to perform full-dimensional feature similarity quantification operations on the semantic feature matrix and each baseline feature matrix in the library, generating a feature similarity matrix that characterizes the degree of similarity across dimensions and regions.
[0064] S05, based on the feature similarity matrix, generates an art feature conflict detection report.
[0065] Optionally, the conflict detection report is an output file used to present the detection results, which may include feature conflict location, classification, quantitative data, and visual representation. The design terminal can perform conflict region extraction, conflict level classification, and conflict spatial correlation analysis on the feature similarity matrix based on preset conflict judgment rules and similarity threshold systems adapted to the norms of the art and design field. Through conflict result analysis and presentation processing methods such as threshold filtering, spatial aggregation, visualization rendering, and topological correlation modeling, it generates and integrates full-dimensional conflict information to obtain an interpretable and traceable art feature conflict detection report.
[0066] In one embodiment, based on image data, a multimodal feature extraction network is used to extract multi-dimensional feature vectors, including:
[0067] S11, Input the image data into the multimodal feature extraction network and perform the following operations:
[0068] S11.1 Extract the pixel space features of the image data to obtain the pixel space vector;
[0069] S11.2 Extract color information from the image based on pixel space vectors, and perform spatial quantization processing on the color information to generate a color distribution statistical matrix;
[0070] S11.3, based on the color distribution statistical matrix, a semantic embedding encoder is used to perform feature semantic mapping to obtain the color semantic feature vector;
[0071] S11.4 Perform instance segmentation on the image data, identify the design element region, and generate a design element mask based on the design element region;
[0072] S11.5 integrates pixel space vectors, color distribution statistics matrix, color semantic feature vectors, and design element masks to obtain multi-dimensional feature vectors.
[0073] For example, the design terminal can input the image data of the target work into a multimodal feature extraction network (which can be a deep learning neural network model pre-trained based on a ResNet backbone network with 4 branches for parallel feature extraction, the 4 branches corresponding to pixel spatial feature extraction, color information quantization, color semantic mapping, and design element instance segmentation, respectively). The design terminal can extract pixel spatial features of the image data through the convolutional layers or residual modules of the multimodal feature extraction network to generate pixel spatial vectors, which represent the low-level structural information of the image (such as pixel edges and pixel gradients). Based on the pixel spatial vectors, the design terminal can extract color information using color histograms or clustering algorithms and perform spatial quantization processing, such as generating a color distribution statistical matrix by dividing the color frequency into blocks to quantify the distribution pattern of color in different regions. The design terminal can use a pre-trained semantic embedding encoder, such as Word2Vec or a variant of BERT, to map the color distribution statistical matrix, converting the color information into a high-dimensional semantic vector, i.e., a color semantic feature vector, to enhance the semantic consistency of color features. The design terminal can also use an instance segmentation model (such as Mask) to extract color information. R-CNN processes image data to identify regions of specific design elements (such as graphics or text) and generates binary masks as design element masks to label the spatial location of elements. The design terminal can integrate pixel spatial vectors, color distribution statistical matrices, color semantic feature vectors, and design element masks through feature splicing or weighted fusion to form a unified multi-dimensional feature vector that fully covers the feature attributes of art design. Through hierarchical extraction and semantic mapping, it solves the problem of insufficient feature quantization in traditional manual detection.
[0074] In one embodiment, the multi-dimensional feature vector is decomposed into a multi-scale feature matrix, including:
[0075] S21, according to the preset scale, perform multi-level feature extraction on the pixel space vector to generate a multi-level feature matrix, which includes a detail layer, a texture layer and a contour layer.
[0076] S22, based on design element masks, uses morphological closing operations to analyze spatial topological relationships;
[0077] S23, based on the color distribution statistical matrix and spatial topological relationship, assigns values to the detail layer, texture layer and contour layer to obtain a multi-scale feature matrix.
[0078] Specifically, the design terminal can perform multi-level feature extraction on pixel spatial vectors according to a preset scale (such as through the multi-layer output of a feature pyramid network or convolutional neural network), generating a multi-level feature matrix containing detail layers, texture layers, and contour layers. The detail layer is used to extract local high-frequency components (such as edge details), the texture layer is used to extract pattern repetition structures, and the contour layer is used to extract overall shape information, ensuring that the extracted feature scale covers both micro and macro scales. Based on the design element mask (generated by instance segmentation), the design terminal can analyze spatial topological relationships using morphological closing operations (such as dilation and erosion operations using circular or rectangular structural elements) to quantify the adjacent, contained, or overlapping relationships between elements, enhancing spatial consistency. The design terminal can combine the color distribution statistical matrix (representing color frequency distribution) and spatial topological relationships to assign values to each layer of the multi-level feature matrix, such as assigning color statistical values as weights to corresponding regions of the detail layer, texture layer, and contour layer, generating a multi-scale feature matrix that integrates color, texture, and spatial information. This step solves the problem of single feature granularity in traditional methods through multi-scale decomposition, providing a foundation for subsequent semantic mapping.
[0079] In one embodiment, the multi-scale feature matrix is mapped to a preset semantic space to obtain a semantic feature matrix, including:
[0080] S31, cluster the feature matrices in the detail layer, texture layer and contour layer of the multi-scale feature matrix according to the feature dimension to generate hierarchical-scale feature sub-blocks;
[0081] S32, channel concatenation of hierarchical-scale feature sub-blocks to generate a fused feature body;
[0082] S33, input the fused feature volume into a pre-trained 3D convolutional neural network to extract multi-layer semantic feature maps;
[0083] S34. Perform cross-scale association calculation on the multi-layer semantic feature map to obtain the semantic feature matrix.
[0084] Optionally, the terminal design can perform feature dimension clustering on the feature matrices within the detail layer, texture layer, and contour layer of the multi-scale feature matrix, such as using K-means or autoencoder clustering algorithms. This groups the feature vectors within each layer based on similarity, generating hierarchical-scale feature sub-blocks, reducing feature redundancy and enhancing local semantic consistency. The terminal design can also perform channel concatenation on these hierarchical-scale feature sub-blocks, integrating sub-blocks from different layers and scales along the channel dimension to generate a fused feature body. This fused feature body retains the complementarity of multi-level information. The terminal design can then input the fused feature body into a pre-trained 3D convolutional neural network. This 3D convolutional neural network captures the spatial and scale-dimensional correlations of features through 3D convolutional kernels, extracting multi-layer semantic feature maps. Lower-level feature maps retain detailed information, while higher-level feature maps encode abstract semantics. Finally, the terminal design can perform cross-scale correlation calculations on the multi-layer semantic feature maps and output a semantic feature matrix using a multi-head attention mechanism. This method, through various deep learning components (such as clustering algorithms, 3D convolutional neural networks, and attention mechanisms), solves the problem of feature semantic inconsistency, improving the objectivity and accuracy of conflict detection.
[0085] In one embodiment, the baseline feature matrix library is constructed in a preset semantic space using labeled feature data of typical art and design works; the dimension of the baseline feature matrix is consistent with that of the semantic feature matrix.
[0086] Calculate the feature similarity between the semantic feature matrix and each benchmark feature matrix in the benchmark feature matrix library to obtain the feature similarity matrix, including:
[0087] S41, based on the semantic feature matrix and the benchmark feature matrix library, a set of positive and negative sample pairs is constructed using a triplet sampling strategy, wherein the anchor sample of the set of positive and negative sample pairs is selected from the semantic feature matrix of the target work;
[0088] S42, perform feature alignment processing on the positive and negative sample pairs to obtain a cross-modal feature set;
[0089] S43. The spatial similarity score of the cross-modal feature set is calculated using the contrastive loss function to obtain the feature similarity matrix.
[0090] Specifically, the design terminal can construct a set of positive and negative sample pairs based on the semantic feature matrix and a benchmark feature matrix library in the preset semantic space (constructed from the labeled data of typical art and design works). In the set of positive and negative sample pairs, anchor samples can be randomly selected from the semantic feature matrix of the target work, positive samples can be selected from the same cluster as the anchor sample (e.g., randomly selecting non-anchor features within the same cluster), and negative samples can be selected from different clusters than the anchor, prioritizing clusters farther from the anchor (e.g., sorting by cluster center distance and selecting features from the farthest cluster), or introducing a hard negative sample mining strategy to select indistinguishable negative samples from adjacent but different clusters to enhance the robustness of the model. The design terminal can perform feature alignment processing on the set of positive and negative sample pairs, such as mapping features of different modalities to a unified dimensional space through layer normalization or linear projection, eliminating scale differences, generating cross-modal feature sets, and achieving feature comparability. The design terminal uses a contrastive loss function (such as InfoNCE or triplet loss) to calculate the spatial similarity score of cross-modal feature sets. This function measures the distance between the anchor point and positive and negative samples, such as using Euclidean distance or dot product similarity, and optimizes the loss to make positive sample pairs closer and negative sample pairs farther apart, outputting a feature similarity matrix. This feature similarity matrix quantifies the degree of similarity between the target work and the benchmark library in various dimensions. This embodiment uses a deep learning component and a triplet design to solve the sample bias problem in traditional comparisons, improving the objectivity of conflict detection.
[0091] In one embodiment, S51, cross-scale correlation calculation is performed on the multi-layer semantic feature map to obtain the semantic feature matrix, using the following formula:
[0092]
[0093] in, Here, M represents the semantic feature matrix, and M represents the number of multi-head attention heads. Let j be the dynamic weight coefficient of the j-th attention head. For the j-th query matrix, Let j be the value matrix of the attention head. Let be the learnable projection matrix of the j-th attention head. For a single attention head feature dimension, This is the channel attention adjustment coefficient. For channel attention function, For aggregated channel features, This is the Sigmoid activation function.
[0094] For example, the terminal design can use the above formula to perform cross-scale correlation calculation on multi-layer semantic feature maps and generate a semantic feature matrix. This formula is used to dynamically fuse multi-scale features and enhance semantic consistency: by combining multi-head attention mechanism and channel attention, weighted aggregation of cross-scale features is achieved, where M attention heads process features in parallel, and the dynamic weight coefficient of each head... By adjusting the contributions at different scales using learnable parameters, the query matrix can be queried. Sum matrix Derived from multi-layer semantic feature maps respectively, and through The function calculates the similarity weights, which are then projected onto the matrix. Mapping to preserve spatial information, channel attention function and It can be based on aggregated channel features Enhance the response of the key channels, and adjust the coefficients accordingly. Balance attention weights using the Sigmoid activation function Output the normalized semantic feature matrix. In this embodiment, The introduction of γ enables the model to adapt to different art and design scenarios, improving feasibility. This formula is closely related to the overall technical features: it is based on multi-layer semantic feature maps, outputs a semantic feature matrix as the basis for subsequent similarity calculations, and compares it with a benchmark feature matrix library to support the objectivity of conflict detection.
[0095] In one embodiment, an art feature conflict detection report is generated based on the feature similarity matrix, including:
[0096] S61, set a similarity threshold based on the feature similarity matrix, and extract regions whose similarity in each dimension exceeds the corresponding similarity threshold from the feature similarity matrix to obtain conflict feature regions;
[0097] S62, Based on each conflict feature region, generate a feature conflict heatmap;
[0098] S63, spatially aggregate the feature conflict heatmaps to generate an aggregated conflict heatmap, and based on the aggregated conflict heatmap, divide the heatmaps into graded heat regions according to the preset heat value grading rules.
[0099] S64, establish a spatial association network based on the spatial topological relationship of each thermal region and each design element mask;
[0100] S65 integrates and aggregates conflict heatmaps, heatmaps of various levels, and spatial correlation networks to generate an art feature conflict detection report.
[0101] Specifically, the design terminal can set a similarity threshold based on the feature similarity matrix. This threshold can be dynamically adjusted according to art and design standards or historical data (e.g., determined by percentiles or statistical distributions). It can then extract regions from the matrix whose similarity exceeds the threshold in each dimension, thus obtaining conflict feature regions. For example, it can locate high-similarity blocks using sliding windows or region growing algorithms to accurately capture conflict regions. The design terminal can also generate feature conflict heatmaps based on each conflict feature region: using color mapping technology (e.g., HSV color space) to convert similarity values into heatmap values (e.g., red represents high conflict, blue represents low conflict, visually presenting the conflict distribution). Furthermore, the design terminal can spatially aggregate the feature conflict heatmaps, generating an aggregated conflict heatmap through weighted averaging or maximum pooling. Based on preset heatmap value grading rules (which can use equal intervals or quantiles), the heatmap values are divided into multiple levels (e.g., high, medium, low conflict), resulting in graded heatmap regions for easier quantitative analysis. The design terminal can establish a spatial association network based on the spatial topological relationships of each thermal region and each design element mask. For example, it can use graph theory methods to treat thermal regions as nodes and the spatial overlap or adjacency of design element masks as edges, calculating network properties (such as degree centrality or connectivity) to reveal the propagation path of conflicts. The design terminal can integrate and aggregate conflict heatmaps, hierarchical thermal regions, and spatial association networks to generate an artistic feature conflict detection report. The report may include visual charts, quantitative data, and text descriptions, and is output through an interactive interface. This embodiment solves the problem of insufficient visualization in traditional manual detection through threshold processing, heatmap rendering, and network modeling, improving the intuitiveness and automation level of conflict detection.
[0102] The aforementioned deep learning-based art and design feature conflict detection method acquires image data of the target artwork and utilizes a multimodal feature extraction network to extract multi-dimensional feature vectors, achieving comprehensive and automated feature extraction of the artwork and overcoming the technical challenge of low efficiency in traditional manual detection. Multi-scale decomposition of the multi-dimensional feature vectors generates a multi-scale feature matrix, enabling quantitative analysis of multi-level features such as detail, texture, and contour layers of design elements, thus resolving the inconsistency problem caused by strong subjectivity. Furthermore, the multi-scale feature matrix is mapped to a preset semantic space to obtain a semantic feature matrix. Feature dimension clustering and a 3D convolutional neural network enhance the semantic consistency of features, improving the objectivity of detection. Based on the semantic feature matrix and a benchmark feature matrix library, a triplet sampling strategy is used to calculate the feature similarity matrix, achieving accurate cross-modal feature comparison and quantitative analysis of complex conflict relationships. By setting a similarity threshold, a feature conflict heatmap is generated and spatially aggregated. Combined with design element masks, a spatial association network is established, outputting a visual detection report. This not only achieves multi-dimensional identification of art and design feature conflicts but also improves the interpretability and automation level of the results through intuitive presentation.
[0103] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.
[0104] Based on the same inventive concept, this application also provides a deep learning-based art design feature conflict detection system for implementing the aforementioned deep learning-based art design feature conflict detection method. The solution provided by this system is similar to the implementation described in the above method. Therefore, the specific limitations of one or more deep learning-based art design feature conflict detection system embodiments provided below can be found in the limitations of the deep learning-based art design feature conflict detection method described above, and will not be repeated here.
[0105] In one exemplary embodiment, such as Figure 2 As shown, a deep learning-based art design feature conflict detection system is provided, including:
[0106] The image data acquisition module 101 is used to acquire image data of the target work, and based on the image data, a multimodal feature extraction network is used to extract multi-dimensional feature vectors.
[0107] The multimodal feature extraction module 102 is used to perform multi-scale decomposition on multi-dimensional feature vectors to obtain multi-scale feature matrices;
[0108] The multi-scale feature decomposition module 103 maps the multi-scale feature matrix to a preset semantic space to obtain a semantic feature matrix;
[0109] The semantic space mapping module 104 is used to calculate the feature similarity between the semantic feature matrix and each benchmark feature matrix in the benchmark feature matrix library, and obtain the feature similarity matrix.
[0110] The conflict report generation module 105 is used to generate an art feature conflict detection report based on the feature similarity matrix.
[0111] In one embodiment, the image data acquisition module 101 can also be used for:
[0112] The image data is input into the multimodal feature extraction network, and the following operations are performed:
[0113] Extract pixel space features from image data to obtain pixel space vectors;
[0114] Color information of an image is extracted based on pixel space vectors, and spatial quantization is performed on the color information to generate a color distribution statistical matrix.
[0115] Based on the color distribution statistical matrix, a semantic embedding encoder is used to perform feature semantic mapping to obtain a color semantic feature vector.
[0116] Perform instance segmentation on the image data, identify design element regions, and generate design element masks based on the design element regions;
[0117] By integrating pixel space vectors, color distribution statistics matrix, color semantic feature vectors, and design element masks, a multi-dimensional feature vector is obtained.
[0118] In one embodiment, the multimodal feature extraction module 102 can also be used for:
[0119] According to the preset scale, multi-level feature extraction is performed on the pixel space vector to generate a multi-level feature matrix, which includes a detail layer, a texture layer and a contour layer.
[0120] Based on the design element mask, spatial topological relationships are analyzed using morphological closing operations.
[0121] Based on the color distribution statistical matrix and spatial topological relationships, values are assigned to the detail layer, texture layer and contour layer to obtain a multi-scale feature matrix.
[0122] In one embodiment, the multi-scale feature decomposition module 103 can also be used for:
[0123] Feature dimension clustering is performed on the feature matrices in the detail layer, texture layer and contour layer of the multi-scale feature matrix to generate hierarchical-scale feature sub-blocks;
[0124] Channel concatenation is performed on hierarchical-scale feature sub-blocks to generate a fused feature volume;
[0125] The fused feature volume is input into a pre-trained 3D convolutional neural network to extract multi-layer semantic feature maps;
[0126] Cross-scale correlation calculations are performed on the multi-layer semantic feature maps to obtain the semantic feature matrix.
[0127] In one embodiment, the baseline feature matrix library in the semantic space mapping module 104 is constructed in a preset semantic space using the labeled feature data of typical art and design works; the dimension of the baseline feature matrix is consistent with that of the semantic feature matrix.
[0128] The semantic space mapping module 104 can also be used for:
[0129] Based on the semantic feature matrix and the benchmark feature matrix library, a set of positive and negative sample pairs is constructed using a triplet sampling strategy, wherein the anchor samples of the set of positive and negative sample pairs are selected from the semantic feature matrix of the target work.
[0130] The positive and negative sample pairs are aligned to obtain a cross-modal feature set;
[0131] The spatial similarity score of the cross-modal feature set is calculated using the contrastive loss function, and the feature similarity matrix is obtained.
[0132] In one embodiment, the conflict report generation module 105 can also be used to:
[0133] A similarity threshold is set based on the feature similarity matrix, and regions whose similarity in each dimension exceeds the corresponding similarity threshold are extracted from the feature similarity matrix to obtain conflict feature regions;
[0134] Based on each conflict feature region, a feature conflict heatmap is generated;
[0135] Spatially aggregate the heatmaps of conflicting features to generate an aggregated conflict heatmap, and based on the aggregated conflict heatmap, divide the heatmap into graded heat regions according to a preset heat value grading rule.
[0136] Based on the spatial topological relationships of each thermal region and each design element mask, a spatial correlation network is established;
[0137] By integrating and aggregating conflict heatmaps, heat regions at various levels, and spatial correlation networks, an art feature conflict detection report is generated.
[0138] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of the deep learning-based art design feature conflict detection method as described above.
[0139] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.
[0140] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.
[0141] The above-described embodiments are merely illustrative of several implementation methods of the embodiments of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the embodiments of this application, and these modifications and improvements all fall within the protection scope of the embodiments of this application.
Claims
1. A method for detecting art design feature conflicts based on deep learning, characterized in that, The method includes: Image data of the target artwork is acquired, and based on the image data, a multimodal feature extraction network is used to extract multidimensional feature vectors. The multi-dimensional feature vectors are decomposed into multi-scale features to obtain multi-scale feature matrices. The multi-scale feature matrix is mapped to a preset semantic space to obtain a semantic feature matrix; Calculate the feature similarity between the semantic feature matrix and each benchmark feature matrix in the benchmark feature matrix library to obtain the feature similarity matrix; Based on the feature similarity matrix, an art feature conflict detection report is generated.
2. The method according to claim 1, characterized in that, Based on the image data, a multimodal feature extraction network is used to extract multidimensional feature vectors, including: The image data is input into the multimodal feature extraction network, and the following operations are performed: Extract the pixel space features from the image data to obtain a pixel space vector; Color information of an image is extracted based on pixel space vectors, and spatial quantization is performed on the color information to generate a color distribution statistical matrix. Based on the color distribution statistical matrix, a semantic embedding encoder is used to perform feature semantic mapping to obtain a color semantic feature vector. The image data is segmented into instances to identify design element regions, and a design element mask is generated based on the design element regions. By integrating the pixel space vector, the color distribution statistics matrix, the color semantic feature vector, and the design element mask, the multi-dimensional feature vector is obtained.
3. The method according to claim 2, characterized in that, The multi-scale decomposition of the multi-dimensional feature vector to obtain the multi-scale feature matrix includes: According to a preset scale, multi-level feature extraction is performed on the pixel space vector to generate a multi-level feature matrix, which includes a detail layer, a texture layer and a contour layer. Based on the design element mask, morphological closing operations are used to analyze spatial topological relationships; Based on the color distribution statistical matrix and the spatial topological relationship, the detail layer, the texture layer and the contour layer are assigned values to obtain the multi-scale feature matrix.
4. The method according to claim 3, characterized in that, The step of mapping the multi-scale feature matrix to a preset semantic space to obtain a semantic feature matrix includes: The feature matrices in the detail layer, texture layer and contour layer of the multi-scale feature matrix are clustered by feature dimension to generate hierarchical-scale feature sub-blocks; The hierarchical-scale feature sub-blocks are channel-cascaded to generate a fused feature body; The fused feature body is input into a pre-trained three-dimensional convolutional neural network to extract multi-layer semantic feature maps; The semantic feature matrix is obtained by performing cross-scale association calculation on the multi-layer semantic feature map.
5. The method according to claim 4, characterized in that, The baseline feature matrix library is constructed in the preset semantic space using labeled feature data of typical art and design works; the dimension of the baseline feature matrix is consistent with that of the semantic feature matrix. The step of calculating the feature similarity between the semantic feature matrix and each benchmark feature matrix in the benchmark feature matrix library to obtain a feature similarity matrix includes: Based on the semantic feature matrix and the benchmark feature matrix library, a set of positive and negative sample pairs is constructed using a triplet sampling strategy, wherein the anchor samples of the set of positive and negative sample pairs are selected from the semantic feature matrix of the target work. The positive and negative sample pairs are subjected to feature alignment processing to obtain a cross-modal feature set; The spatial similarity score of the cross-modal feature set is calculated using a contrastive loss function to obtain the feature similarity matrix.
6. The method according to claim 4, characterized in that, The semantic feature matrix is obtained by performing cross-scale association calculation on the multi-layer semantic feature map using the following formula: in, Here, M represents the semantic feature matrix, and M represents the number of multi-head attention heads. Let j be the dynamic weight coefficient of the j-th attention head. For the j-th query matrix, Let j be the value matrix of the attention head. Let be the learnable projection matrix of the j-th attention head. For a single attention head feature dimension, This is the channel attention adjustment coefficient. For channel attention function, For aggregated channel features, This is the Sigmoid activation function.
7. The method according to claim 5, characterized in that, The step of generating an art feature conflict detection report based on the feature similarity matrix includes: Based on the feature similarity matrix, a similarity threshold is set, and regions whose similarity in each dimension exceeds the corresponding similarity threshold are extracted from the feature similarity matrix to obtain conflict feature regions; Based on each of the conflict feature regions, a feature conflict heatmap is generated; The heatmaps of each feature conflict are spatially aggregated to generate an aggregated conflict heatmap, and based on the aggregated conflict heatmap, hierarchical heat regions are divided according to a preset heat value hierarchical rule. A spatial association network is established based on the spatial topological relationship between the thermal regions and the design element masks. By integrating the aggregated conflict heatmap, each of the hierarchical heat regions, and the spatial association network, the art feature conflict detection report is generated.
8. A deep learning-based art design feature conflict detection system, characterized in that, The system includes: The image data acquisition module is used to acquire image data of the target work, and based on the image data, to extract multi-dimensional feature vectors using a multimodal feature extraction network. A multimodal feature extraction module is used to perform multi-scale decomposition on the multi-dimensional feature vector to obtain a multi-scale feature matrix; A multi-scale feature decomposition module is used to map the multi-scale feature matrix to a preset semantic space to obtain a semantic feature matrix. The semantic space mapping module is used to calculate the feature similarity between the semantic feature matrix and each benchmark feature matrix in the benchmark feature matrix library, and obtain the feature similarity matrix. The conflict report generation module is used to generate an art feature conflict detection report based on the feature similarity matrix.
9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.