Design scheme generation method based on information retrieval technology

Abstract

The invention discloses a design scheme generation method based on an information retrieval technology, and belongs to the field of information retrieval and image-text generation. Collecting the text description and an image matched with the content of the text description, preprocessing the text description and the image respectively, and performing feature extraction processing to obtain extracted visual feature monk and semantic features: extracting the visual features in the image data; for character description data matched with the image, processing the character image by using a semantic segmentation network so as to identify and segment a character region, extracting uniformly distributed semantic feature points through a quadtree distribution algorithm, and integrating the semantic feature points of all the regions to form a complete semantic feature point set corresponding to the image data including characters; a multi-scale feature map is constructed by using a feature pyramid network, the extracted visual features and semantic features are mapped to a unified feature space in the multi-scale feature map, and it is ensured that key information of the respective features is reserved in the mapping process.

Description

Technical Field

[0001] The invention relates to the fields of information retrieval and image and text generation, and in particular to a design scheme generation method based on information retrieval technology. Background Art

[0002] In the creative industry, art design optimization plays a vital role. It not only affects the creativity and aesthetics of the work, but is also a key factor in determining whether the work can be successful in the market. However, traditional art design optimization methods are often limited by the designer's personal experience and creativity, making it difficult to produce high-quality design solutions in a short period of time.

[0003] Therefore, seeking an efficient and automated method for optimizing art design solutions has become a hot topic in current research.

[0004] As an important branch of computer science, information retrieval technology, with its powerful information retrieval and information processing capabilities, provides new possibilities for the optimization of art design schemes. By utilizing information retrieval technology, designers can quickly find design cases that are similar or related to user needs from massive design resources, thereby providing inspiration and reference for designers. At the same time, information retrieval technology can also help designers automatically evaluate and optimize design schemes, thereby improving design efficiency and quality. Therefore, the research on design scheme generation methods based on information retrieval technology has important theoretical and practical significance. Summary of the invention

[0005] In view of the shortcomings of the existing technology, a design solution generation method based on information retrieval technology is provided. It integrates RAG technology and LLM large model, making it possible to quickly retrieve and generate design solutions highly relevant to user needs from a huge design database, and uses automated tools to quickly evaluate and iterate the design, significantly improving the quality of design and the speed of innovation.

[0006] To achieve the above technical objectives, the method for generating an art design scheme based on information retrieval technology of the present invention comprises the following steps:

[0007] Step 1: collect text descriptions and images matching the text descriptions as original solutions, associate the two original solutions and store them in databases, preprocess the text description database and the image database, remove non-keywords in the text descriptions, and unify the image parameters;

[0008] Step 2: Perform feature extraction on the image preprocessing data to obtain visual features and semantic features: Use a natural encoder to extract visual features from image data; For text description data matching the image, use a semantic segmentation network to process the text image to identify and segment the text area, extract evenly distributed semantic feature points through a quadtree allocation algorithm, integrate the semantic feature points of all areas, and form a complete set of semantic feature points corresponding to the image data including the text;

[0009] Step 3: Use feature pyramid network (FPN) to construct multi-scale feature map, use multi-scale feature map to fuse feature maps of different levels, and simultaneously capture rich semantic features and visual feature context information; use transformation matrix to map the visual features and semantic features extracted in the previous step to the unified feature space in the multi-scale feature map, and ensure that the key information of each feature is retained during the mapping process; use attention mechanism to weight different visual features and semantic features to achieve the fusion of visual features and semantic features, and this fusion process can fully reflect the core visual and semantic attributes of the original design scheme; based on the image description generation task pre-training end-to-end model, the task pre-training end-to-end model is a traditional encoder-decoder architecture, and the fused semantic features and visual features are input into the task pre-training end-to-end model for optimization to obtain fused features, and the fused features of all texts and images constitute a material database;

[0010] Step 4. Use a search engine to obtain design materials related to the required fusion features from the design material database as needed; block the design materials by fixed-size blocks to obtain multiple block information; use the Transformers library to load the BGE v1.5 model, and then use the Momentum Encoder method to fine-tune the BGE v1.5 model to convert the block information into a high-dimensional vector. According to the preset number of clusters K, use the K-means clustering quantization method to initialize K cluster centers, and assign each high-dimensional vector to the nearest cluster center to form K clusters; use InvertedFile (IVF) to review the index structure of PQ to organize the quantized high-dimensional vectors; construct an inverted list for each cluster to obtain K inverted lists, which contain identifiers of all high-dimensional vectors in the cluster; use the folding tree strategy to optimize the retrieval process; obtain a candidate set based on the similarity between the embedded information and the node vector in the vector database, and the node vector with the highest similarity is the required design material; finally, use the context enrichment method to expand the required design material to obtain the expanded design material;

[0011] Input the expanded design materials into the LLM language model to obtain a preliminary text design plan; summarize and extract key points of the preliminary text design plan to obtain semantic key points, and finally input the semantic key points into the existing text-born image model to obtain a preliminary picture design plan, that is, the required image product; integrate the preliminary text design plan and the preliminary picture design plan to obtain a preliminary design plan;

[0012] Step 5: Based on the review information provided by the user, determine whether the preliminary design solution needs to be optimized. The process of optimizing the preliminary design solution is to return to iterative steps 1 to 4 based on the materials that have been searched.

[0013] Furthermore, the natural encoder includes an input layer Layer 1, hidden layers Layer 2, Layer 3 and an output layer Layer 4, the input layer is used to receive image data and convert it into a form that the neural network can understand; the hidden layer maps the input image data to a low-micro space through the ReLU nonlinear activation function; the output layer is used to compare the output of the hidden layer with the input of the input layer, calculate the loss through the mean square error function, and then adjust the weights and biases in the network according to the error calculated by the loss function through the back propagation algorithm; the loss calculation and back propagation process are repeated until a predetermined number of iterations are reached, and then the visual features are output;

[0014] The input data is transformed through the natural encoder to output the desired visual features; the first layer has two neurons x1, x2 and an intercept term c1; the second layer has two neurons y1, y2 and an intercept term c2; the third layer is the output, with two neurons h1 and h2; the value on each line represents the weight of the connection between neurons, and the loss function σ uses the Sigmoi function;

[0015] For image data including text, a semantic segmentation network is used to process the text image to identify and segment the text area; a quadtree allocation algorithm is used to recursively segment the image until a preset minimum segmentation unit is reached; in each leaf node area of ​​the quadtree, an adaptive extraction threshold algorithm is applied to determine the extraction threshold of feature points; according to the set threshold, evenly distributed semantic feature points are extracted from each leaf node area; the semantic feature points of all areas are integrated to form a complete set of semantic feature points corresponding to the image data including text;

[0016] For pure text data, that is, text descriptions, natural language processing technology is used to extract text features of text descriptions. Text descriptions can provide more key information for the material search process, and integrate semantic feature point sets and text features to obtain semantic features.

[0017] Furthermore, the backbone of the feature pyramid network (FPN) uses ResNet, including a bottom-up circuit and a top-down circuit, and the two circuits are connected horizontally; bottom-up: in fact, it is the forward process of the network; in the forward process, selection is performed in multiples of 2; the size of the feature map will change after passing through certain layers, but will not change when passing through other layers. The layers that do not change the size of the feature map are grouped into one stage, and each extracted feature is the output of the last layer of each stage, so that a feature pyramid can be formed; the backbone of the feature selection network is scaled in multiples of 2; top-down: upsampling is used in the process, and the upsampling here is a 2-fold upsampling. After upsampling, the height and width are guaranteed to be the same, and the horizontal addition and fusion operation can be performed; the horizontal connection is to fuse the upsampling result with the feature map of the same size generated from the bottom up; after fusion, a 3*3 convolution kernel is used to convolve each fusion result to eliminate the aliasing effect of upsampling.

[0018] Furthermore, the importance of different features is weighted by using the attention mechanism to achieve the fusion of visual features and semantic features. This fusion process can fully reflect the core visual and semantic attributes of the original design scheme. The attention mechanism weighting process includes three parts. The first part is the feature extractor. First, the visual features are sent to the pre-trained CNN to extract N rasterized features X = {x1, x2, ..., xN}; the second part is the encoder. The semantic features and rasterized features are sent to the encoder respectively to obtain the encoded features. The encoder is composed of L encoding layers with cross-scale feature fusion self-attention; the third part is the decoder. The encoded features are sent to the decoder to obtain the decoded description sequence. The decoder is composed of L decoding layers with ordinary self-attention. The decoded description sequence is the fused semantic features and visual features.

[0019] Furthermore, the BGE v1.5 model fine-tuned by the Transformers framework is as follows:

[0020] The BGE v1.5 model is loaded using the Transformers library, and then the Momentum Encoder method is used to fine-tune the BGEv1.5 model. The Momentum Encoder method is used to introduce a momentum update mechanism for dynamic updating into the BGE v1.5 model to better capture dynamic changes in the data; MoCo using the Momentum Encoder method is used to enhance representation learning through dynamic dictionary and momentum update; BGE has an animation function, which adds dynamic elements to the BGE v1.5 model. The art design task needs to show the changes of the BGE v1.5 model under different conditions. The fine-tuned BGE v1.5 model can better adapt to art design tasks and create design works that are both beautiful and practical.

[0021] Furthermore, the Inverted File (IVF) is used to review the index structure of PQ to organize the quantized vectors. The IVF index distributes data points into multiple "inverted lists", where each cluster represents a visual vocabulary and is represented by the cluster center. That is, an inverted list is constructed for each cluster, resulting in K inverted lists, which contain the identifiers of all vectors in the cluster.

[0022] For each cluster center, an inverted list is constructed, which contains the identifiers of all vectors belonging to the cluster; this means that each inverted list stores a set of similar vectors, thereby accelerating the approximate nearest neighbor search. When searching, the IVF index first calculates the distance between the query vector and all cluster centers, then selects the K nearest cluster centers, and evaluates the vectors in the inverted lists corresponding to these cluster centers; the IVF index is suitable for large-scale data sets, especially in scenarios where fast approximate search is required. It significantly improves query efficiency by reducing the search space and using cluster centers as index nodes; the indexing method combining composite IVF and PQ can increase the search speed by 16.5 times without sacrificing accuracy, and the overall performance is improved by 92 times.

[0023] Furthermore, a folding tree strategy is used to optimize the retrieval process;

[0024] According to the constructed inverted list, starting from the root node, the traversal direction is determined according to the similarity between the embedded information and the node vector in the vector database. In each layer, the subtree most similar to the embedded information is selected to continue to go deeper;

[0025] During the traversal process, the vector of each node encountered is similar to the embedded information based on the cosine similarity algorithm; according to the similarity threshold or ranking, a candidate set is determined, which contains the node vectors that are most similar to the embedded information;

[0026] The calculation formula for cosine similarity is:

[0027]

[0028] Where p and q are two vectors, p i and q i are the values ​​of the i-th dimension of vectors p and q respectively. The range of cosine similarity is [-1,1]. The closer the value is to 1, the more similar the two vectors are, and the closer the value is to -1, the less similar the two vectors are. Since the folding tree strategy flattens all levels into a single layer for retrieval, all your nodes will be flattened to the same layer at a time, making it easier to quickly query them using the approximate nearest neighbor ANN library.

[0029] If the user feedback indicates that no modification is required, there is no need to optimize the preliminary design solution;

[0030] If the review information fed back by the user indicates that modification is required, the modification design ideas fed back by the user are obtained, the design modification ideas are used to replace the original design plan in step 1, and steps 1-4 are re-executed until the review information fed back by the user indicates that no modification is required.

[0031] Beneficial effects: The design solution generation method based on information retrieval technology of the present invention has greater advantages than traditional methods. This method has made better optimizations in terms of functionality and practicality of the design solution. This method can better improve the aesthetic performance and visual appeal of the design solution. This method has better overall effects when multiple aspects are comprehensively considered. This application explores a new method for optimizing art design solutions by integrating RAG technology and the LLM large model; these technologies make it possible to quickly retrieve and generate design solutions that are highly relevant to user needs from a huge design database. Through this method, designers can access a wide range of design inspiration and resources, while using automated tools to quickly evaluate and iterate designs, significantly improving the quality of design and the speed of innovation. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 is a flow chart of a design solution generation method based on information retrieval technology in an embodiment of the present invention;

[0033] Figure 2 is a schematic diagram of the structure of a natural encoder recorded in an embodiment of the present invention;

[0034] Figure 3 It is a schematic diagram of the characteristic pyramid network structure recorded in an embodiment of the present invention;

[0035] Figure 4 It is a schematic diagram of the weighted flow of the attention mechanism recorded in an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0037] like Figure 1 As shown, the present invention discloses a design solution generation method based on information retrieval technology, the steps are as follows:

[0038] Step 1: Obtain an original design scheme, and preprocess the original design scheme to obtain preprocessed data;

[0039] The pre-processing process includes the steps of: data cleaning and data arrangement;

[0040] 1.1. Data cleaning: For text data, we first use a hash table to identify and delete duplicate entries to achieve the purpose of removing duplicate items, then use regular expressions to remove irrelevant characters, and then use a predefined stop word list to remove stop words, and finally use the Chinese word segmentation algorithm to further perform word segmentation;

[0041] 1.2. For image data, first, the images are uniformly adjusted to the specified format and size to ensure the consistency of subsequent processing; then, the denoising algorithm is applied to remove the noise in the image to improve the image quality; then, the OpenCV library is used to perform image enhancement processing on the image, such as contrast adjustment, sharpening, etc., to improve the visual effect of the image; finally, the threshold segmentation method is used to perform segmentation operations on the pre-processed image to divide the image into multiple regions or objects for further analysis or processing.

[0042] 1.3. Regarding data sorting and processing: according to the preset encoding standard, use library functions to perform unified text encoding processing on text data;

[0043] 1.4. Perform uniform resolution processing on image data according to the preset target resolution, and use nearest neighbor interpolation to optimize the image quality of the image data after resolution processing.

[0044] Step 2: Perform feature extraction on the preprocessed data to obtain visual features and semantic features;

[0045] 2.1. For image data, use natural encoder to extract visual features from image data;

[0046] The natural encoder includes an input layer, a hidden layer, and an output layer. The input layer is used to receive image data and convert it into a form that the neural network can understand; the hidden layer maps the input image data to a low-micro space through the ReLU nonlinear activation function; the output layer is used to compare the output of the hidden layer with the input of the input layer, calculate the loss through the mean square error function, and then adjust the weights and biases in the network according to the error calculated by the loss function through the back propagation algorithm; the loss calculation and back propagation process are repeated until the predetermined number of iterations is reached, and then the visual features are output.

[0047] For more information about natural encoders, see Figure 2 , Layer1 represents the input layer, Layer 2 and Layer 3 represent hidden layers, and Layer 4 represents the output layer. The input data is transformed through the natural encoder to output the desired visual features. The first layer has two neurons x1 and x2 and an intercept term c1. The second layer has two neurons y1 and y2 and an intercept term c2. The third layer is the output, with two neurons h1 and h2. The value on each line represents the weight of the connection between neurons, and the loss function σ uses the Sigmoi function.

[0048] Furthermore, for image data including text, a semantic segmentation network is used to process the text image to identify and segment the text area; a quadtree allocation algorithm is used to recursively segment the image until a preset minimum segmentation unit is reached; in each leaf node area of ​​the quadtree, an adaptive extraction threshold algorithm is applied to determine the extraction threshold of feature points; according to the set threshold, evenly distributed semantic feature points are extracted from each leaf node area; the semantic feature points of all areas are integrated to form a complete set of semantic feature points corresponding to the image data including text;

[0049] 2.2. For pure text data, i.e. text description, natural language processing technology is used to extract text features of the text description;

[0050] 2.3. Integrate the semantic feature point set and text features to obtain semantic features.

[0051] Step 3: Perform feature fusion processing on the visual features and semantic features to obtain fusion features;

[0052] 3.1. Use Feature Pyramid Network (FPN) to construct multi-scale feature maps. The constructed multi-scale feature maps can fuse feature maps at different levels to capture rich contextual information.

[0053] Specifically, the backbone of FPN uses ResNet, including a bottom-up circuit and a top-down circuit, and the two circuits are connected horizontally; bottom-up: in fact, it is the forward process of the network; in the forward process, selection is performed according to integer multiples of 2; the size of the feature map will change after passing through some layers, but will not change when passing through other layers. The layers that do not change the size of the feature map are classified as a stage, and each extracted feature is the output of the last layer of each stage, so that a feature pyramid can be formed; the backbone of the feature selection network is scaled by integer multiples of 2; top-down: upsampling is used in the process, where the upsampling here is a 2-fold upsampling, which ensures that the height and width are the same after upsampling, and the horizontal addition and fusion operation can be performed; the 2-fold upsampling here is implemented by a simple neighboring interpolation algorithm; and the horizontal connection is to fuse the upsampling result with the feature map of the same size generated from the bottom up; after fusion, a 3*3 convolution kernel will be used to convolve each fusion result to eliminate the aliasing effect of upsampling.

[0054] For an example, see Figure 3 , taking the input image of 640x 640x 3 as an example, Figure 3 It can be seen that after the fusion of the new feature map, a 3x3 convolution layer will be connected, that is, further fusion, to obtain P2, P3, P4, P5, and P5 is downsampled to obtain P6.

[0055] 3.2. Based on the transformation matrix, the visual features and semantic features extracted in the previous step are mapped into a unified feature space to ensure that the key information of each feature is retained during the mapping process;

[0056] 3.3. Use the attention mechanism to weight the importance of different features to achieve the fusion of visual features and semantic features. This fusion process can fully reflect the core visual and semantic attributes of the original design solution;

[0057] See also Figure 4 , Figure 4It is a weighted process of the attention mechanism, which mainly includes three parts. The first part is the feature extractor, which first sends the visual features into the pre-trained CNN to extract N rasterized features X = {x1, x2, ..., xN}; the second part is the encoder, which sends the semantic features and rasterized features into the encoder respectively to obtain the encoded features. The encoder is composed of L encoding layers with cross-scale feature fusion self-attention; the third part is the decoder, which sends the encoded features into the decoder to obtain the decoded description sequence. The decoder is composed of L decoding layers with ordinary self-attention; the decoded description sequence is the fused semantic features and visual features.

[0058] 3.4. Pre-train an end-to-end model based on the image description generation task, input the fused semantic features and visual features into the aforementioned model for optimization, and obtain fused features; the end-to-end model is an encoder-decoder architecture, which allows the model to be trained in an end-to-end manner and generate descriptions directly from images without the need for intermediate human intervention.

[0059] Step 4: Generate a preliminary design plan based on the fusion features and retrieval enhanced generation technology;

[0060] 4.1. Based on the fusion features, use the search engine to obtain relevant design materials from the design material database;

[0061] 4.2. According to the preset segmentation length and the preset overlap number, the design material is segmented into blocks in a fixed size manner to obtain multiple block information;

[0062] 4.3. Fine-tune the BGE v1.5 model using the Transformers framework to obtain a fine-tuned embedding model. Based on the fine-tuned embedding model, convert the block information into a high-dimensional vector to obtain embedded information, and store the embedded information in a vector database.

[0063] Specifically, the Transformers library is used to load the BGE v1.5 model, and then the Momentum Encoder method is used to fine-tune the BGE v1.5 model. The Momentum Encoder method introduces a momentum update mechanism to dynamically update the parameters of the model to better capture the dynamic changes in the data; MoCo is a typical example that enhances representation learning through dynamic dictionaries and momentum updates; BGE has an animation function that can add dynamic elements to the model. Art design tasks may require showing the changes of the model under different conditions. The fine-tuned BGE model can increase the attractiveness of the design, that is, the fine-tuned BGE model can better adapt to art design tasks and create design works that are both beautiful and practical.

[0064] 4.4. Use K-means clustering quantization technology to convert the embedded information into a more manageable representation, that is, according to the preset number of clusters K, use K-means clustering quantization method to initialize K cluster centers; assign each high-dimensional vector to the nearest cluster center to form K clusters;

[0065] 4.5. Use Inverted File (IVF) to review the index structure of PQ to organize the quantized vectors. IVF indexing is a clustering-based indexing method that distributes data points into multiple "inverted lists". Each cluster represents a visual vocabulary and can be represented by the cluster center. That is, an inverted list is constructed for each cluster, and K inverted lists are obtained, which contain the identifiers of all vectors in the cluster.

[0066] Specifically, for each cluster center, an inverted list is constructed, which contains the identifiers of all vectors belonging to the cluster; this means that each inverted list stores a set of similar vectors, thereby accelerating the approximate nearest neighbor search. When searching, the IVF index first calculates the distance between the query vector and all cluster centers, then selects the K nearest cluster centers, and evaluates the vectors in the inverted lists corresponding to these cluster centers; the IVF index is suitable for large-scale data sets, especially in scenarios that require fast approximate search. It significantly improves query efficiency by reducing the search space and using cluster centers as index nodes; and the indexing method combining composite IVF and PQ can increase the search speed by 16.5 times without sacrificing accuracy, and the overall performance is improved by 92 times.

[0067] 4.6. Use the folding tree strategy to optimize the retrieval process. Based on the constructed inverted list, start from the root node and determine the traversal direction based on the similarity between the embedded information and the node vector in the vector database. In each layer, select the subtree that is most similar to the embedded information to continue to go deeper.

[0068] During the traversal process, the vector of each node encountered is similar to the embedded information based on the cosine similarity algorithm; according to the similarity threshold or ranking, a candidate set is determined, which contains the node vectors that are most similar to the embedded information;

[0069] The calculation formula for cosine similarity is:

[0070]

[0071] Where p and q are two vectors, p i and q i are the values ​​of the i-th dimension of vectors p and q respectively,

[0072] The range of cosine similarity is [-1, 1]. The closer the value is to 1, the more similar the two vectors are, and the closer the value is to -1, the less similar the two vectors are.

[0073] Unlike tree traversal, the collapsed tree strategy flattens all levels into a single layer for retrieval. This means that all nodes are flattened to the same level for fast querying using the ANN library.

[0074] 4.7. Use a cross encoder to sort the node vectors in the candidate set to obtain the required design materials;

[0075] The cross encoder calculates the similarity between queries and items (such as documents or paragraphs) by jointly encoding them, and then sorts each node vector according to the similarity, and outputs the node vector with the highest similarity as the required design material.

[0076] 4.8. Use context enrichment methods to expand the demand design materials and obtain expanded design materials;

[0077] Context enrichment methods are a technique used in natural language processing that enhances the understanding of language by considering the words surrounding a word or phrase; context enrichment methods are often used in tasks such as word sense disambiguation, machine translation, question answering systems, and text summarization to improve the model's sensitivity to language nuances and its ability to handle complex language structures.

[0078] 4.9. Input the expanded design materials into the LLM language model to obtain a preliminary text design plan;

[0079] 4.10. Use natural language processing (NLP) technology to summarize and extract key points of the preliminary text design plan to obtain semantic key points;

[0080] 4.11. Input the semantic key points into the text graph model to obtain a preliminary picture design plan;

[0081] The text-based graph model is an artificial intelligence technology that can generate corresponding images based on the text description provided by the user. Specifically, the semantic points are input into the text-based graph model, and the model will analyze these points and generate a preliminary picture design plan. This process usually involves deep learning and image generation algorithms, which enable computers to understand text content and create visual expressions.

[0082] 4.12. Integrate the preliminary text design plan and the preliminary picture design plan to obtain a preliminary design plan.

[0083] Step 5: Obtain the review information of user feedback, and determine whether the preliminary design solution needs to be optimized based on the review information;

[0084] Specific:

[0085] 5.1. If the user feedback indicates that no modification is required, there is no need to optimize the preliminary design plan;

[0086] 5.2. If the review information fed back by the user indicates that modification is required, obtain the design modification ideas fed back by the user, use the design modification ideas to replace the original design plan in step 1, and re-execute steps 1-4 until the review information fed back by the user indicates that no modification is required.

[0087] Comparative test:

[0088] In order to verify the effectiveness of the art design optimization method based on information retrieval technology, 20 design schemes were selected as experimental samples, 10 of which were optimized using traditional design methods, and the other 10 were optimized using design methods based on information retrieval technology. Then, these design schemes were evaluated in terms of creativity, practicality, and aesthetics, and compared with traditional methods in groups one by one, so as to analyze the advantages and disadvantages of this method.

[0089]

[0090] (1) The experimental results clearly show that the art design optimization method based on information retrieval technology has higher advantages than the traditional method. In terms of creativity, the average score of the design method based on information retrieval technology reached 0.80, which is significantly higher than the 0.69 of the traditional method. This shows that the method based on information retrieval technology has obvious advantages in stimulating the innovation and originality of design solutions.

[0091] (2) In terms of practicality, the average score of the design method based on information retrieval technology is 0.78, slightly higher than the 0.72 of the traditional method. This shows that the design method based on information retrieval technology has made better optimization in terms of functionality and practicality of the design solution.

[0092] (3) In terms of aesthetics, the average score of the design method based on information retrieval technology is 0.83, which is significantly higher than the 0.73 of the traditional method. This shows that the method based on information retrieval technology can better improve the aesthetic performance and visual appeal of the design scheme.

[0093] (4) In terms of comprehensive score, the average comprehensive score of the design method based on information retrieval technology is 0.78, slightly higher than the 0.72 of the traditional method. This shows that the design method based on information retrieval technology has a better overall effect after comprehensive consideration of multiple aspects.

[0094] Therefore, through comparative analysis of experimental data, it can be concluded that the design method based on information retrieval technology shows obvious advantages in creativity, practicality and aesthetics, and is more suitable for the optimization of art design schemes.

Claims

1. A design solution generation method based on information retrieval technology, characterized in that: Here are the steps: Step 1: collect text descriptions and images matching the text descriptions as original solutions, associate the two original solutions and store them in databases, preprocess the text description database and the image database, remove non-keywords in the text descriptions, and unify the image parameters; Step 2: Perform feature extraction on the image preprocessing data to obtain visual features and semantic features: use a natural encoder to extract visual features from image data; For the text description data matched with the image, the text image is processed using a semantic segmentation network to identify and segment the text area, and the evenly distributed semantic feature points are extracted through a quadtree allocation algorithm, and the semantic feature points of all areas are integrated to form a complete set of semantic feature points corresponding to the image data including the text; Step 3: Use the Feature Pyramid Network (FPN) to construct a multi-scale feature map, and use the multi-scale feature map to fuse feature maps at different levels to simultaneously capture rich semantic features and visual feature context information; Using the conversion matrix, the visual features and semantic features extracted in the previous step are mapped to the unified feature space in the multi-scale feature map to ensure that the key information of each feature is retained during the mapping process; using the attention mechanism to weight different visual features and semantic features to achieve the fusion of visual features and semantic features. This fusion process can fully reflect the core visual and semantic attributes of the original design scheme; based on the image description generation task pre-training end-to-end model, the task pre-training end-to-end model is a traditional encoder-decoder architecture. The fused semantic features and visual features are input into the task pre-training end-to-end model for optimization to obtain fused features, and the fused features of all texts and images constitute a material database; Step 4. Use a search engine to obtain design materials related to the required fusion features from the design material database as needed; block the design materials by fixed-size blocks to obtain multiple block information; use the Transformers library to load the BGE v1.5 model, and then use the Momentum Encoder method to fine-tune the BGE v1.5 model to convert the block information into a high-dimensional vector. According to the preset number of clusters K, use the K-means clustering quantization method to initialize K cluster centers, and assign each high-dimensional vector to the nearest cluster center to form K clusters; use InvertedFile (IVF) to review the index structure of PQ to organize the quantized high-dimensional vectors; construct an inverted list for each cluster to obtain K inverted lists, which contain identifiers of all high-dimensional vectors in the cluster; use the folding tree strategy to optimize the retrieval process; obtain a candidate set based on the similarity between the embedded information and the node vector in the vector database, and the node vector with the highest similarity is the required design material; finally, use the context enrichment method to expand the required design material to obtain the expanded design material; Input the expanded design materials into the LLM language model to obtain a preliminary text design plan; summarize and extract key points of the preliminary text design plan to obtain semantic key points, and finally input the semantic key points into the existing text-born graph model to obtain a preliminary picture design plan, that is, the required image product; Integrate the preliminary text design plan and the preliminary picture design plan to obtain a preliminary design plan; Step 5: Based on the review information provided by the user, determine whether the preliminary design solution needs to be optimized. The process of optimizing the preliminary design solution is to return to iterative steps 1 to 4 based on the materials that have been searched.

2. The design solution generation method based on information retrieval technology according to claim 1 is characterized in that: The encoder includes an input layer Layer 1, hidden layers Layer 2, Layer 3, and an output layer Layer 4. The input layer is used to receive image data and convert it into a form that the neural network can understand; the hidden layer maps the input image data to a low-micro space through the ReLU nonlinear activation function; the output layer is used to compare the output of the hidden layer with the input of the input layer, calculate the loss through the mean square error function, and then adjust the weights and biases in the network according to the error calculated by the loss function through the back propagation algorithm; the loss calculation and back propagation process are repeated until the predetermined number of iterations is reached, and then the visual features are output; Through the natural encoder, the input data is transformed to output the desired visual features; The first layer has two neurons x1, x2 and an intercept term c1; the second layer has two neurons y1, y2 and an intercept term c2; the third layer is the output, with two neurons h1 and h2; the value on each line represents the weight of the connection between neurons, and the loss function σ uses the Sigmoi function; For image data including text, a semantic segmentation network is used to process the text image to identify and segment the text area; a quadtree allocation algorithm is used to recursively segment the image until a preset minimum segmentation unit is reached; in each leaf node area of ​​the quadtree, an adaptive extraction threshold algorithm is applied to determine the extraction threshold of feature points; according to the set threshold, evenly distributed semantic feature points are extracted from each leaf node area; the semantic feature points of all areas are integrated to form a complete set of semantic feature points corresponding to the image data including text; For pure text data, that is, text descriptions, natural language processing technology is used to extract text features of text descriptions. Text descriptions can provide more key information for the material search process, and integrate semantic feature point sets and text features to obtain semantic features.

3. The design solution generation method based on information retrieval technology according to claim 1 is characterized in that: The backbone of the Feature Pyramid Network (FPN) uses ResNet, which includes a bottom-up circuit and a top-down circuit, and the two circuits are connected horizontally; bottom-up: in fact, it is the forward process of the network; in the forward process, selection is made according to integer multiples of 2; the size of the feature map will change after passing through some layers, but will not change when passing through other layers. The layers that do not change the size of the feature map are grouped into one stage, and each extracted feature is the output of the last layer of each stage, so that a feature pyramid can be formed; the backbone of the feature selection network is scaled by integer multiples of 2; Top-down: Upsampling is used in the process. The upsampling here is a 2-fold upsampling. After upsampling, the height and width are guaranteed to be the same, and the horizontal addition and fusion operation can be performed; the horizontal connection is to fuse the upsampling result with the feature map of the same size generated from the bottom up; after fusion, a 3*3 convolution kernel is used to convolve each fusion result to eliminate the aliasing effect of upsampling.

4. The design solution generation method based on information retrieval technology according to claim 3 is characterized in that: The importance of different features is weighted by the attention mechanism to achieve the fusion of visual features and semantic features. The fusion process can fully reflect the core visual and semantic attributes of the original design scheme. The attention mechanism weighting process includes three parts. The first part is the feature extractor. First, the visual features are sent to the pre-trained CNN to extract N rasterized features X = {x1, x2, ..., xN}; the second part is the encoder. The semantic features and rasterized features are sent to the encoder respectively to obtain the encoded features. The encoder is composed of L encoding layers with cross-scale feature fusion self-attention; the third part is the decoder. The encoded features are sent to the decoder to obtain the decoded description sequence. The decoder is composed of L decoding layers with ordinary self-attention. The decoded description sequence is the fused semantic features and visual features.

5. The design solution generation method based on information retrieval technology according to claim 1 is characterized in that: The BGE v1.5 model fine-tuned by the Transformers framework is as follows: Use the Transformers library to load the BGE v1.5 model, and then use the Momentum Encoder method to fine-tune the BGE v1.5 model. Use the Momentum Encoder method to introduce a momentum update mechanism for dynamic updates into the BGE v1.5 model to better capture dynamic changes in the data; use MoCo of the Momentum Encoder method to enhance representation learning through dynamic dictionaries and momentum updates; BGE has animation function, which adds dynamic elements to the BGE v1.5 model. Art design tasks need to show the changes of the BGE v1.5 model under different conditions. Through fine-tuning, the BGE v1.5 model can better adapt to art design tasks and create design works that are both beautiful and practical.

6. The design solution generation method based on information retrieval technology according to claim 1 is characterized in that: The Inverted File (IVF) is used to review the index structure of PQ to organize the quantized vectors. The IVF index distributes data points into multiple "inverted lists", where each cluster represents a visual word and is represented by the cluster center. That is, an inverted list is constructed for each cluster, and K inverted lists are obtained, which contain the identifiers of all vectors in the cluster. For each cluster center, an inverted list is constructed, which contains the identifiers of all vectors belonging to the cluster; this means that each inverted list stores a set of similar vectors, thereby accelerating the approximate nearest neighbor search. When searching, the IVF index first calculates the distance between the query vector and all cluster centers, then selects the K nearest cluster centers, and evaluates the vectors in the inverted lists corresponding to these cluster centers; the IVF index is suitable for large-scale data sets, especially in scenarios where fast approximate search is required. It significantly improves query efficiency by reducing the search space and using cluster centers as index nodes; the indexing method combining composite IVF and PQ can increase the search speed by 16.5 times without sacrificing accuracy, and the overall performance is improved by 92 times.

7. The design solution generation method based on information retrieval technology according to claim 6 is characterized in that: Use the folding tree strategy to optimize the retrieval process; According to the constructed inverted list, starting from the root node, the traversal direction is determined according to the similarity between the embedded information and the node vector in the vector database. In each layer, the subtree most similar to the embedded information is selected to continue to go deeper; During the traversal process, the vector of each node encountered is similar to the embedded information based on the cosine similarity algorithm; according to the similarity threshold or ranking, a candidate set is determined, which contains the node vectors that are most similar to the embedded information; The calculation formula for cosine similarity is: Where p and q are two vectors, p i and q i are the values ​​of the i-th dimension of vectors p and q respectively. The range of cosine similarity is [-1,1]. The closer the value is to 1, the more similar the two vectors are, and the closer the value is to -1, the less similar the two vectors are. Since the folding tree strategy flattens all levels into a single layer for retrieval, all your nodes will be flattened to the same layer at a time, making it easier to quickly query them using the approximate nearest neighbor ANN library.

8. The design solution generation method based on information retrieval technology according to claim 6 is characterized in that: If the user feedback indicates that no modification is required, there is no need to optimize the preliminary design solution; If the review information fed back by the user indicates that modification is required, the modification design ideas fed back by the user are obtained, the design modification ideas are used to replace the original design plan in step 1, and steps 1-4 are re-executed until the review information fed back by the user indicates that no modification is required.

Images