Product effect picture generation method based on entropy weight optimal sequence graph and stable diffusion model

Abstract

The invention discloses a product effect picture generation method based on an entropy weight optimal sequence graph and a stable diffusion model. The method comprises the following steps: 1) performing multi-image sentiment analysis and right confirmation; 2) analyzing and drawing a product sketch; 3) generating a multi-image product effect picture scheme; and 4) evaluating a multi-image product effect picture scheme, selecting the effect picture with the highest comprehensive score as a final scheme, and performing three-dimensional presentation. Case verification is carried out by taking a new energy automobile as an example, it is proved that the proposed method can comprehensively and accurately generate a product effect picture scheme conforming to the multi-image emotion of the user, the design efficiency and quality of the product effect picture and the user satisfaction degree can be improved, new vitality is injected for innovation in the field of product generation and design, and the method is worthy of popularization and application. And an intelligent design system tool for performing high-efficiency design based on user requirements is provided for designers, and the design quality and productivity of product generation design are improved.

Description

Technical Field

[0001] The invention relates to the field of product generation design, and in particular to an automatic generation design and evaluation method for a multi-image product effect diagram scheme. Background Art

[0002] Product design usually includes: 2D design sketch, 3D modeling, 2D rendering, that is, the design process of "2D-3D-2D". Driven by the wave of informatization and emotionalization, the extensive application of technology based on emotional big data is profoundly changing the face of product design. [1] Big data provides a vast amount of user sentiment and design information resources for the product design field, including a wealth of diversified data such as online product reviews and images, providing designers with unprecedented innovation space and providing strong support for optimizing traditional design processes and efficiency. [2] . More and more designers are beginning to focus on injecting rich user emotional information into products in a more efficient and intelligent design method, making the products more emotional and personalized. Under this trend, multi-image product renderings are a key link in the product design process. The accuracy, diversity and efficiency of their design are crucial to meeting the growing emotional needs of consumers. However, since it is difficult to fully and accurately determine multi-image emotions, the design analysis accuracy is insufficient, and there is a lack of effective intelligent design methods for multi-image emotional communication.

[0003] References:

[0004] [1]DING M,YUAN Y,ZHANG X,et al.Product color emotional design based on deep learning[J].ComputerIntegratedManufacturing System,2023,29(5):1647.

[0005] [2][1]Aravindan S,Ekbal A.Feature Extraction and Opinion Mining inOnline Product Reviews[C] / / International Conference on InformationTechnology.IEEE,2015.DOI:10.1109 / ICIT.2014.72. Summary of the invention

[0006] The purpose of the present invention is to address the deficiencies in the prior art, solve the problem that the multi-image emotions are difficult to determine comprehensively and accurately in product generation design, resulting in insufficient analysis accuracy, and lack of effective intelligent design methods for multi-image emotion communication, and propose a new method for multi-image product rendering generation design based on the priority graph-entropy weight method and the stable diffusion model under big data. The method constructs a comprehensive multi-image product rendering generation design integrated method architecture, including four stages: multi-image emotion analysis and confirmation, product sketch analysis and drawing, multi-image product rendering scheme generation and multi-image product rendering scheme evaluation. It can enrich the methodological system of product design at the theoretical level, and provide designers with more efficient and accurate generative artificial intelligence design tools at the practical level, while also bringing consumers richer, more personalized and high-quality product choices.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions:

[0008] An automatic generation design and evaluation method for a multi-image product rendering scheme includes the following steps:

[0009] Step S1: Multi-image sentiment analysis and confirmation;

[0010] Step S2: product sketch analysis and drawing;

[0011] Step S3: generating a multi-image product rendering scheme;

[0012] Step S4: Evaluation of multi-image product rendering schemes.

[0013] Specifically, the content of the multi-image sentiment analysis and confirmation in step S1 is:

[0014] Step S11: Constructing a perceptual dictionary

[0015] Collect user comment texts through web crawlers, and after data cleaning, word segmentation and part-of-speech tagging, use Jieba word segmentation technology to tag the online comment text data, obtain all adjectives, remove invalid or similar adjectives, classify them, list the network relationships between adjectives, use TF-IDF technology to calculate the TF-IDF values ​​of the adjectives obtained and sort them from high to low according to the values, and select positive adjectives with high TF-IDF values ​​to construct a perceptual dictionary;

[0016] Step S12: Initial weight calculation based on priority graph method

[0017] Step S121: Determine the scoring criteria for each indicator

[0018] Take the scores of u experts as rows and n evaluation indicators as columns, and calculate the average value S of each indicator. j :

[0019]

[0020] In the above formula: S j represents the average value of index j, j = 1, 2, ..., n; S ji represents the scoring result of expert i on the jth indicator, i = 1, 2, …, u;

[0021] Step S122: Compare the average values

[0022] Compare the two indicators, the larger one is scored as 1 point, the smaller one is scored as 0 point, and if they are completely equal, each is scored as 0.5 point, so as to obtain the judgment matrix A; where a ij is 0.5 or 0 or 1;

[0023]

[0024] Step S123: Calculate the weight p of each indicator j :

[0025]

[0026] In the above formula: p j is the weight of index j; a ij is the value of the i-th row and j-th column in the judgment matrix;

[0027] Step S13: Multiple image rights confirmation based on entropy weight method

[0028] Step S131: Data standardization

[0029] The original data is standardized, and the evaluation index is set to the perceptual image vocabulary obtained above, the number of evaluation indexes is set to n, and the number of evaluation objects is set to m, where: j = 1, 2, 3 ..., n; i = 1, 2, 3 ..., m;

[0030] Since all the indicators involved are positive indicators, we only need to use the positive indicator processing formula:

[0031]

[0032] In the above formula, Z ij is the positive index value of the jth indicator of the i-th evaluation object after processing, x ij It is the positive index value of the jth indicator of the ith evaluation object before processing.

[0033] Step S132: Calculate information entropy and information effective value

[0034] The information entropy value formula of the jth indicator is:

[0035]

[0036] In the above formula: is the proportion of the jth indicator of the i-th evaluation object to the total value of the indicator; ln∫ ij is the value of the jth indicator of the evaluation object i;

[0037] The information effect value of the jth indicator is:

[0038] d j =1-e j (6)

[0039] Step S133: Calculate the weight of each indicator

[0040] Calculate the indicator weights. The weight of the jth indicator is:

[0041]

[0042] The weights obtained by the two methods are normalized to obtain the specific multi-image weights, and the formula is as follows:

[0043]

[0044] In the above formula, p and q are the weights obtained by two methods.

[0045] Specifically, the content of the product sketch analysis and drawing in step S2 is:

[0046] Firstly, the key styling elements that affect the overall styling image of the product are determined through focus group interviews, and the morphological analysis method is used to extract the styling elements from the selected product morphology, so as to construct an element sample set as the basis for fuzzy comprehensive evaluation; then, the user's multi-image emotion is used as the evaluation index of fuzzy comprehensive evaluation, and the fuzzy comprehensive evaluation method is used to evaluate the element sample set, and the element sample with the highest comprehensive score is used as the sketch image board, and the designer then draws the sketch based on the sketch image board;

[0047] The specific process of the fuzzy comprehensive evaluation is as follows:

[0048] Step S21: Determine the index set and weight vector

[0049] The index set is I = {x1, x2, ... x p}, that is, the user emotional image extracted in step S1; the weight vector is The weight vector is a comprehensive weight obtained by the priority graph method and the entropy weight method;

[0050] Step S22: Create a comment set V

[0051] S comments constitute a comment set, denoted as V = {v1, v2, ... vs};

[0052] Step S23: Establish a single index evaluation vector

[0053] The evaluation matrix R is obtained by integrating the evaluation vectors of various indicators. ij ) p*s , where r ij is the evaluation result of the i-th indicator, and p is the number of indicators;

[0054] Step S24: Synthesize the fuzzy comprehensive evaluation result vector

[0055] Use appropriate operators to synthesize W and the evaluation matrix R to obtain the fuzzy comprehensive evaluation result vector A of the evaluated sample:

[0056]

[0057] Specifically, the steps of generating the multi-image product rendering scheme in step S3 are as follows:

[0058] Step S31: Contour extraction

[0059] The Canny model in ControlNet is used to extract the contour of the input sketch, and then the extracted contour is input into the rendering generation model, so as to accurately control the generation of the rendering;

[0060] The network structure of the ControlNet consists of two parts, namely locked copy and trainable copy; the locked copy part fixes the original weights of the stable diffusion model to ensure that the image generation ability of the model remains unchanged during the subsequent training process, thereby providing a stable performance for the model; and the parameters of the trainable copy part are initialized to the parameters of the corresponding trainable copy, which will be updated and optimized in the subsequent training process of ControlNet, so that the entire network learning is adjusted to the task specified by the user;

[0061] like Figure 2 As shown, Figure 2 (a) is the output before the stable diffusion model. Figure 2 The difference between (b) and (a) is that the ControlNet structure is added; zero convolution is a 1x1 convolution layer, and the weights and biases are initialized to 0. Before the training starts, since the zero convolution initializes the weights and biases to 0, the entire ControlNet model is still the original stable diffusion model; only after the training data is added, Figure 2The trainable copy in (b) and the learnable parameter values ​​in zero convolution will change, so that the entire network learning can be adjusted to the task specified by the user.

[0062] Step S32: Embedding LoRA model into stable diffusion model

[0063] like Figure 3 As shown in the figure, the stable diffusion model is used as the big framework of the whole effect picture generation model, and the LoRA model that maps the emotional image of the target user is embedded in it to form the effect picture generation model;

[0064] Step S33: training the large model for generating effect pictures to obtain the large model for generating effect pictures with target emotional tendency, inputting the hand-drawn sketches of designers into the trained large model for generating effect pictures to generate product effect picture solutions in batches;

[0065] The stable diffusion model continuously adds noise to the image during training to destroy the image. During inference, the noisy image is denoised to restore the original image. During the T iterations of the training process, x at any time t It can be represented by x0 and β. x0 at any time can also be represented by x t And the noise z is obtained:

[0066]

[0067] In the above formula, a and β represent the noise intensity;

[0068]

[0069] In the above formula, x0 is the initial clean image, and the noisy image x is generated by adding noise z t , the second formula is transformed from the first formula, which means that the noise image is subtracted from a certain intensity of noise to obtain the image x0;

[0070] The obtained noisy image x t Applied to the reverse diffusion stage of the model;

[0071] In the reverse diffusion stage, the reverse process obeys the Gaussian distribution. Given the original clear image, the reverse process can be converted into a forward process through the Bayesian formula, and then the reverse process distribution can be calculated. The formula is expressed as:

[0072]

[0073] In the above formula, it means that given x0, the inverse process obeys the Gaussian distribution, and the Bayesian formula is used to convert the inverse process into the forward process;

[0074]

[0075] In the above formula, x t and x0 can be converted to each other, and the mean can also be obtained from x t After subtracting different noises, the product rendering scheme can be obtained through the reverse diffusion stage and additional parameter control;

[0076] In order to generate high-quality product renderings, the optimization of the model in the training phase includes:

[0077] Fitting distribution

[0078] The model estimation results follow a Gaussian distribution:

[0079]

[0080] In the above formula, the mean formula includes x t and noise, due to x t It is known during training that we only need to get the estimated noise of the model output, which can be obtained by the model using x t and t are estimated; in the reverse diffusion stage, the generated result of the previous step is input, the estimated noise of each step in the reverse diffusion stage is calculated, and then the noise is subtracted from the input to obtain the output of the current time step;

[0081] Reverse Process Distribution

[0082] Compare the inverse process distribution with the estimated distribution of the model. Since the KL divergence can be used to describe the difference between the two distributions and the KL divergence of the multivariate Gaussian distribution has a closed-form solution, it is deduced that the loss function becomes the KL divergence of the two Gaussian distributions, expressed as:

[0083] L t =D KL (q(x t |x t+1 ,x0)||p θ (x t |x t+1 )); 1≤t≤T-1 (15)

[0084] In the above formula, q distribution is the inverse process distribution, p is the model estimation distribution, and the training loss is the KL divergence of the two Gaussian distributions, that is, the gap between the two distributions;

[0085] Taking the above trained stable diffusion model as the basis, LoRA adapts to the downstream tasks by only training some new parameters according to the data of the corresponding downstream tasks. After the new parameters are trained, the new parameters are merged with the old model parameters by using the re-parameterization method, which can not only achieve the effect of fine-tuning the entire model on the new task, but also will not increase the inference time during inference.

[0086] Specifically, the content of the multi-image product rendering scheme evaluation in step S4 is:

[0087] The TOPSIS method is used to evaluate the product rendering schemes with multi-image emotions as the evaluation index, combined with the weights obtained by the priority diagram-entropy weight method, and the scheme with the highest score is selected as the final scheme to ensure that the design best meets the user's emotional imagery;

[0088] The calculation process of the TOPSIS method is as follows:

[0089] Step S41: Calculate the comprehensive weight h according to the priority graph-entropy weight method j Establish the weighted matrix V with the standard matrix R, and determine the positive ideal solution (V + ) and the negative ideal solution (V - ):

[0090] V=h j × ij (16)

[0091]

[0092] Step S42: Calculate the distance between each evaluation object and the positive ideal solution and the negative ideal solution:

[0093]

[0094] Step S43: Calculate the closeness C between the evaluation object and the ideal solution i :

[0095]

[0096] In the above formula: C i The larger the value is, the closer the i-th rendering is to the optimal level. i The value range is from 0 to 1;

[0097] The TOPSIS method is used to screen out the product rendering scheme that best meets multiple images, and the designer conducts subsequent design activities based on the rendering scheme.

[0098] Compared with the prior art, the present invention has the following beneficial effects:

[0099] 1. The present invention proposes a new method for calculating user multi-image weights with finer granularity. In order to solve the ambiguity and uncertainty problems of traditional multi-image recognition, the present invention proposes a new method for user multi-image weight determination based on the combination technology of the preference graph and the entropy weight method under big data. The preference graph method is a subjective weighting method, which is easy to operate, easy to understand, and has reliable results. The entropy weight method is an objective weighting method with good stability and reliability. Combining the preference graph method with the entropy weight method not only has objective advantages, but also can reflect the degree of importance that decision makers attach to different indicators, which helps to improve the objectivity and accuracy of user multi-image weight determination.

[0100] 2. A new method for generating stable diffusion product renderings that incorporates a multi-image weight calculation mechanism has been formed. In response to the cumbersome traditional product design expression process of "2D-3D-2D", the present invention constructs an intelligent generation model for product renderings based on an adjustable multi-image weight stable diffusion model. This model uses advanced computer vision and machine learning technology to directly start from a two-dimensional design sketch, and through a stable diffusion algorithm with adjustable multi-image weights, it can intelligently generate high-quality product renderings, thereby achieving the automatic generation of high-quality design scheme renderings and more accurate satisfaction of users' diverse emotional needs with design schemes, thereby improving the efficiency, quality and user satisfaction of product design expression. BRIEF DESCRIPTION OF THE DRAWINGS

[0101] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present disclosure and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying creative work.

[0102] Figure 1 is a flow chart of the method of the present invention;

[0103] Figure 2 It is a structural comparison diagram of the stable diffusion model of the present invention;

[0104] Figure 3 is a structural diagram of the stable diffusion model of the present invention;

[0105] Figure 4 is a schematic diagram of the LoRA model of the present invention;

[0106] Figure 5 is a schematic diagram of the network relationship of the perceptual adjectives of the present invention;

[0107] Figure 6 is a front view of a car sample in an embodiment of the present invention;

[0108] Figure 7is an example diagram of key modeling elements in an embodiment of the present invention;

[0109] Figure 8 is a sample set of 60 elements in an embodiment of the present invention;

[0110] Fig. 9 It is a schematic diagram of the sketch image signboard and the types of arrangement and combination in the embodiment of the present invention;

[0111] Fig.10 is a schematic diagram of a designer's creative hand-drawn sketch scheme in an embodiment of the present invention;

[0112] Fig.11 is a schematic diagram of a rendering generation model in an embodiment of the present invention;

[0113] Fig.12 It is a schematic diagram summarizing the hand-drawn sketches and generated effect diagrams of the designer in the embodiment of the present invention;

[0114] Fig.13 This is a rendering of the best solution in the embodiment of the present invention;

[0115] Fig.14 It is a three-dimensional engineering modeling diagram of the optimal solution in the embodiment of the present invention. DETAILED DESCRIPTION

[0116] In order to facilitate those of ordinary skill in the art to understand and implement the present invention, the following detailed description is given to each step of the method proposed by the present invention, and it should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms also fall within the scope defined by the appended claims of the present application.

[0117] Example

[0118] like Figure 1 As shown, the present invention provides a method for automatically generating, designing and evaluating a multi-image product rendering scheme, comprising the following steps:

[0119] Step S1: Multi-image sentiment analysis and confirmation;

[0120] Step S2: product sketch analysis and drawing;

[0121] Step S3: generating a multi-image product rendering scheme;

[0122] Step S4: Evaluation of multi-image product rendering schemes.

[0123] Specifically, the content of the multi-image sentiment analysis and confirmation in step S1 is:

[0124] Step S11: Constructing a perceptual dictionary

[0125] Collect user comment texts through web crawlers, and after data cleaning, word segmentation and part-of-speech tagging, use Jieba word segmentation technology to tag the online comment text data, obtain all adjectives, remove invalid or similar adjectives, classify them, list the network relationships between adjectives, use TF-IDF technology to calculate the TF-IDF values ​​of the adjectives obtained and sort them from high to low according to the values, and select positive adjectives with high TF-IDF values ​​to construct a perceptual dictionary;

[0126] Step S12: Initial weight calculation based on priority graph method

[0127] Step S121: Determine the scoring criteria for each indicator

[0128] Take the scores of u experts as rows and n evaluation indicators as columns, and calculate the average value S of each indicator. j :

[0129]

[0130] In the above formula: S j represents the average value of index j, j = 1, 2, ..., n; S ji represents the scoring result of expert i on the jth indicator, i = 1, 2, …, u;

[0131] Step S122: Compare the average values

[0132] Compare the two indicators, the larger one is scored as 1 point, the smaller one is scored as 0 point, and if they are completely equal, each is scored as 0.5 point, so as to obtain the judgment matrix A; where a ij is 0.5 or 0 or 1;

[0133]

[0134] Step S123: Calculate the weight p of each indicator j :

[0135]

[0136] In the above formula: p j is the weight of index j; a ij is the value of the i-th row and j-th column in the judgment matrix;

[0137] Step S13: Multiple image rights confirmation based on entropy weight method

[0138] Step S131: Data standardization

[0139] The original data is standardized, and the evaluation index is set to the perceptual image vocabulary obtained above, the number of evaluation indexes is set to n, and the number of evaluation objects is set to m, where: j = 1, 2, 3 ..., n; i = 1, 2, 3 ..., m;

[0140] Since all the indicators involved are positive indicators, we only need to use the positive indicator processing formula:

[0141]

[0142] In the above formula, Z ij is the positive index value of the jth indicator of the i-th evaluation object after processing, x ij It is the positive index value of the jth indicator of the ith evaluation object before processing.

[0143] Step S132: Calculate information entropy and information effective value

[0144] The information entropy value formula of the jth indicator is:

[0145]

[0146] In the above formula: is the proportion of the jth indicator of the i-th evaluation object to the total value of the indicator; ln∫ ij is the value of the jth indicator of the evaluation object i;

[0147] The information effect value of the jth indicator is:

[0148] d j =1-e j (6)

[0149] Step S133: Calculate the weight of each indicator

[0150] Calculate the indicator weights. The weight of the jth indicator is:

[0151]

[0152] The weights obtained by the two methods are normalized to obtain the specific multi-image weights, and the formula is as follows:

[0153]

[0154] In the above formula, p and q are the weights obtained by two methods.

[0155] Specifically, the content of the product sketch analysis and drawing in step S2 is:

[0156] Firstly, the key styling elements that affect the overall styling image of the product are determined through focus group interviews, and the morphological analysis method is used to extract the styling elements from the selected product morphology, so as to construct an element sample set as the basis for fuzzy comprehensive evaluation; then, the user's multi-image emotion is used as the evaluation index of fuzzy comprehensive evaluation, and the fuzzy comprehensive evaluation method is used to evaluate the element sample set, and the element sample with the highest comprehensive score is used as the sketch image board, and the designer then draws the sketch based on the sketch image board;

[0157] The specific process of the fuzzy comprehensive evaluation is as follows:

[0158] Step S21: Determine the index set and weight vector

[0159] The index set is I = {x1, x2, ... x p}, that is, the user emotional image extracted in step S1; the weight vector is The weight vector is a comprehensive weight obtained by the priority graph method and the entropy weight method;

[0160] Step S22: Create a comment set V

[0161] S comments constitute a comment set, denoted as V = {v1, v2, ... v s};

[0162] Step S23: Establish a single index evaluation vector

[0163] The evaluation matrix R is obtained by integrating the evaluation vectors of various indicators. ij ) p*s , where r ij is the evaluation result of the i-th indicator, and p is the number of indicators;

[0164] Step S24: Synthesize the fuzzy comprehensive evaluation result vector

[0165] Use appropriate operators to synthesize W and the evaluation matrix R to obtain the fuzzy comprehensive evaluation result vector A of the evaluated sample:

[0166]

[0167] Specifically, the steps of generating the multi-image product rendering scheme in step S3 are as follows:

[0168] Step S31: Contour extraction

[0169] The Canny model in ControlNet is used to extract the contour of the input sketch, and then the extracted contour is input into the rendering generation model, so as to accurately control the generation of the rendering;

[0170] The network structure of the ControlNet consists of two parts, namely locked copy and trainable copy; the locked copy part fixes the original weights of the stable diffusion model to ensure that the image generation ability of the model remains unchanged during the subsequent training process, thereby providing a stable performance for the model; and the parameters of the trainable copy part are initialized to the parameters of the corresponding trainable copy, which will be updated and optimized in the subsequent training process of ControlNet, so that the entire network learning is adjusted to the task specified by the user;

[0171] Step S32: Embedding LoRA model into stable diffusion model

[0172] The stable diffusion model is used as the framework of the entire large model for effect image generation, and the LoRA model that maps the emotional image of the target user is embedded in it to form the large model for effect image generation.

[0173] Step S33: training the large model for generating effect pictures to obtain the large model for generating effect pictures with target emotional tendency, inputting the hand-drawn sketches of designers into the trained large model for generating effect pictures to generate product effect picture solutions in batches;

[0174] The overall idea of ​​the stable diffusion model is: during training, noise is continuously added to the image to destroy the image, and during inference, the noisy image is denoised to restore the original image. During the T iterations of the training process, x at any time t can be represented by x0 and β, and x0 at any time can also be represented by x t And the noise z is obtained:

[0175]

[0176] In the above formula, a and β represent the noise intensity;

[0177]

[0178] In the above formula, x0 is the initial clean image, and the noisy image x is generated by adding noise z t , the second formula is transformed from the first formula, which means that the noise image is subtracted from a certain intensity of noise to obtain the image x0;

[0179] The obtained noisy image x t Applied to the reverse diffusion stage of the model;

[0180] In the reverse diffusion stage, the reverse process obeys the Gaussian distribution. Given the original clear image, the reverse process can be converted into a forward process through the Bayesian formula, and then the reverse process distribution can be calculated. The formula is expressed as:

[0181]

[0182] In the above formula, it means that given x0, the inverse process obeys the Gaussian distribution, and the Bayesian formula is used to convert the inverse process into the forward process;

[0183]

[0184] In the above formula, x t and x0 can be converted to each other, and the mean can also be obtained from x t After subtracting different noises, the product rendering scheme can be obtained through the reverse diffusion stage and additional parameter control;

[0185] In order to generate high-quality product renderings, the optimization of the model in the training phase includes:

[0186] Fitting distribution

[0187] The model estimation results follow a Gaussian distribution:

[0188]

[0189] In the above formula, the mean formula includes x t and noise, due to x t It is known during training that we only need to get the estimated noise of the model output, which can be obtained by the model using x t and t are estimated; in the reverse diffusion stage, the generated result of the previous step is input, the estimated noise of each step in the reverse diffusion stage is calculated, and then the noise is subtracted from the input to obtain the output of the current time step;

[0190] Reverse Process Distribution

[0191] Compare the inverse process distribution with the estimated distribution of the model. Since the KL divergence can be used to describe the difference between the two distributions and the KL divergence of the multivariate Gaussian distribution has a closed-form solution, it is deduced that the loss function becomes the calculation of the KL divergence of the two Gaussian distributions:

[0192] L t =D KL (q(x t |x t+1 ,x0)||p θ (x t |x t+1 )); 1≤t≤T-1 (15)

[0193] In the above formula, q distribution is the inverse process distribution, p is the model estimation distribution, and the training loss is the KL divergence of the two Gaussian distributions, that is, the gap between the two distributions;

[0194] Based on the above trained stable diffusion model, LoRA only trains some new parameters to adapt to the downstream task according to the data of the corresponding downstream task. After the new parameters are trained, the new parameters are merged with the old model parameters by re-parameterization, which can achieve the effect of fine-tuning the entire model on the new task without increasing the inference time. The schematic diagram of LoRA is shown below. Figure 4 shown. Figure 4 In the figure are the pre-trained model parameters. LoRA adds two structures A and B next to the pre-trained model structure. The parameters of these two structures are initialized to Gaussian distribution and 0, respectively. The additional parameters are 0 at the beginning of training; the input dimension of A and the output dimension of B are the same as the input and output dimensions of the original model, respectively, and are a value much smaller than the input and output dimensions of the original model.

[0195] Specifically, the content of the multi-image product rendering scheme evaluation in step S4 is:

[0196] The TOPSIS method is used to evaluate the product rendering schemes with multi-image emotions as the evaluation index, combined with the weights obtained by the priority diagram-entropy weight method, and the scheme with the highest score is selected as the final scheme to ensure that the design best meets the user's emotional imagery;

[0197] The calculation process of the TOPSIS method is as follows:

[0198] Step S41: Calculate the comprehensive weight h according to the priority graph-entropy weight method j Establish the weighted matrix V with the standard matrix R, and determine the positive ideal solution (V + ) and the negative ideal solution (V - ):

[0199] V=h j × ij (16)

[0200]

[0201] Step S42: Calculate the distance between each evaluation object and the positive ideal solution and the negative ideal solution:

[0202]

[0203] Step S43: Calculate the closeness between the evaluation object and the ideal solution (C i ):

[0204]

[0205] In the above formula: C i The larger the value is, the closer the i-th rendering is to the optimal level. i The value range is from 0 to 1;

[0206] The TOPSIS method is used to select the product rendering scheme that best meets multiple images, and the designer conducts subsequent design activities based on the rendering scheme

[0207] Taking new energy vehicles as an example, the method of the present invention is further explained in combination with the attached drawings and the product effect diagram design process, so as to verify the effectiveness of the method proposed by the present invention.

[0208] Step S1: Multi-image sentiment analysis and confirmation

[0209] The new energy vehicle review data from China Autohome website (https: / / www.autohome.com.cn) is used as the source of user emotional imagery. In order to improve data diversity and accuracy, the review data of 20 popular new energy vehicles in the economy market A-class are selected for crawling, focusing on appearance evaluation. After deduplication, a total of 28,863 reviews were obtained. In view of the fact that the reviews obtained by web crawlers often contain noisy data, preprocessing work such as data cleaning, word segmentation and part-of-speech tagging is first performed. The part-of-speech tagging of online review data is performed using the stuttering word segmentation to obtain all adjectives, remove and classify invalid or similar adjectives, and list the network relationships between adjectives, such as Figure 5 The obtained adjectives are sorted by TF-IDF, and the top 9 adjectives and their TF-IDF are shown in Table 1 below.

[0210] Table 1. Preliminary perceptual image vocabulary

[0211]

[0212]

[0213] The above 9 preliminary perceptual image words were used as evaluation indicators to make a questionnaire, and the subjects were invited to evaluate the influence of the 9 preliminary perceptual image words on the overall styling image of the car. The questionnaire was scored using a Likert 5-point scale, where 1 point means no influence, 2 points means little influence, 3 points means average, 4 points means relatively influential, and 5 points means influential. A total of 51 responses from product design majors were collected, including 30 undergraduates and 21 postgraduates.

[0214] The Cronbach alpha coefficient of the questionnaire calculated by SPSS software was 0.834, indicating that the questionnaire had high reliability. The questionnaire was tested by KMO and Bartlett, and the results are shown in Table 2. The data was spherically distributed, indicating that the data passed the sphericity test.

[0215] Table 2. KMO and Bartlett test

[0216]

[0217] Construct a priority diagram weight table, as shown in Table 3. Calculate the average value of each analysis item, and use the average value to compare two items. The larger the average value, the higher the importance and the higher the weight.

[0218] Table 3. Calculation table of priority diagram weights

[0219]

[0220]

[0221] Combined with the priority diagram weight calculation table, sum each row of data to get the TTL value (index score), and normalize the TTL value to get the weight value. According to Table 4 below, select the top five perceptual image words with the highest weight value for the next step of calculation, which are "fashion", "smooth", "exquisite", "simple", and "texture".

[0222] Table 4. Calculation results of priority graph weights

[0223]

[0224] Invite 20 experts in related professions to form a focus group. The gender ratio of the group is 1:1 (male to female), including 10 university teachers engaged in design, 5 experts in the field of industrial design research, and 5 users with 3 years of consumption experience. 10 car renderings with significant differences were selected from the car renderings data set as samples to distribute questionnaires, and invite subjects to evaluate the degree of conformity between the pictures and 5 preliminary perceptual image words. The questionnaire was scored using the Likert 5-point scale. A total of 51 questionnaire data from product design subjects were collected. The entropy weight method was used to redistribute the weights of the above 5 perceptual image words. The weight distribution results of each indicator calculated by the entropy weight method are shown in Table 5 below.

[0225] Table 5. Calculation results of entropy weight method

[0226]

[0227]

[0228] Table 5 shows that the weights of emotional images are in the following order from large to small: exquisite, simple, textured, smooth, and fashionable. The results of normalizing the weights obtained by the two methods are shown in Table 6 below.

[0229] Table 6. Final weights

[0230]

[0231] Step S2: Product sketch analysis and drawing

[0232] like Figure 6 As shown in the figure, the main views of the above 20 new energy vehicles are selected as case references. In order to eliminate brand and color interference, the samples are converted to grayscale mode and unified in size. The focus group interviews show that Figure 7 As shown in Figure 1, the side profile of the car body, the side window contour line and the headlight contour line are the three key styling elements of the car. Taking the above 20 new energy vehicles as the source, the side profile of the car body, the side window contour line and the headlight contour line of the sample are extracted from them according to the morphological analysis method, and a total of 60 element sample sets of 3 key styling element categories are constructed, as shown in Figure 1. Figure 8 As shown, Figure 8 The left side of the middle represents the body side view contour line sample, the middle represents the side window contour line sample, and the right side represents the headlight contour line sample.

[0233] Find the element samples that match the user's emotional imagery most to build a hand-drawn sketch image board to guide designers in drawing car sketches. Use fuzzy comprehensive evaluation to score the 60 element samples in the element sample set, and set the 5 perceptual image words obtained above as evaluation indicators, namely "fashion", "smooth", "exquisite", "simple" and "texture". The top two scoring element samples in each category become specific reference samples. Use the comprehensive weight obtained by the priority diagram-entropy weight method as the weight vector.

[0234] The fuzzy vectors of the three element categories and the specific scores of each sample are shown below. From Tables 8, 10, and 12 below, it can be concluded that the highest scores in the fuzzy comprehensive evaluation calculation results of the body side contour line samples are samples 8 and 13, the highest scores in the fuzzy comprehensive evaluation calculation results of the side window contour line samples are samples 7 and 8, and the highest scores in the fuzzy comprehensive evaluation calculation results of the headlight contour line samples are samples 6 and 9. The above 6 samples are used as hand-drawn sketch image boards.

[0235] Table 7. Vehicle side contour line - fuzzy vector

[0236]

[0237] Table 8. Fuzzy comprehensive evaluation calculation results of body side contour line samples

[0238]

[0239] Table 9. Side window contour line - blur vector

[0240]

[0241] Table 10. Fuzzy comprehensive evaluation calculation results of side window contour samples

[0242]

[0243] Table 11. Headlamp outline - blur vector

[0244]

[0245] Table 12. Calculation results of fuzzy comprehensive evaluation of headlamp contour samples

[0246]

[0247] like Fig. 9 As shown in the figure, the designer draws the main view sketch of the car based on the features of one body side contour line sample, one side window contour line sample and one headlight contour line sample in the hand-drawn sketch image board. According to the six specific reference samples in the three element categories, the following eight arrangement results can be obtained by arranging and combining them. The eight arrangement results are respectively assigned to eight senior industrial designers, who draw the main view sketch of the car. The results are shown in the figure below. Fig.10 As shown. According to the hand-drawn sketches of designers, it can be concluded that hand-drawn sketches will have different sketch styles due to different designers, which are diverse and innovative.

[0248] Step S3: Generate multi-image product renderings

[0249] The quality of images generated by the stable diffusion model is closely related to the quality of the dataset images, so it is necessary to construct an image dataset consisting of high-quality renderings of the main view of new energy vehicles. Images of common new energy vehicles are crawled from the China Autohome website and formed into a rendering dataset after batch processing; the web crawler keywords used are "new energy vehicles" and "main view".

[0250] To ensure that the image features of the training dataset are clear, this study processed the crawled images. First, duplicate images and images with unclear features were manually removed; second, Photoshop software was used to uniformly adjust them to 512*512 pixels, adjust the car position in the center, and convert them to jpg format. Finally, an original dataset containing 800 car renderings was constructed, laying the foundation for subsequent research.

[0251] Table 13. Example of effect map dataset

[0252]

[0253] Based on the five emotional image words, the corresponding rendering LoRA database was constructed. Through focus group interviews, 100 car renderings that best matched each emotional image were selected from 800 renderings to form the LoRA database of each rendering. Then, using data enhancement technology, the images were mirrored, and finally 5 rendering LoRA databases containing 200 renderings were constructed for the training of the LoRA model embedded in the stable diffusion model.

[0254] First, the images of the five emotional image databases were preprocessed, that is, the images were labeled using Deepbooru. Then the obtained label text and database were packaged uniformly on the basic large model for training, and five embedded effect map LoRa models were generated, which together constituted the effect map generation large model. The training parameters are shown in Table 14.

[0255] Table 14. LoRa model training parameters

[0256]

[0257] Use the trained rendering generation model to generate car renderings. Fig.11 As shown in the figure, the generation of the rendering is the result of the comprehensive control of the sketch scheme, 5 embedded rendering LoRA models that map emotional images, and text descriptions. The control weight of the embedded rendering LoRA model is set to the weight required by the corresponding emotional image in the weight calculation stage. The purpose of adding text descriptions is to make the generated renderings more detailed and of higher quality, and to avoid the generation of some unreasonable images.

[0258] First, the Canny model in Controlnet is used to extract the contours of the designer's creative hand-drawn sketches. The extracted contours are used as the input of the model to accurately control the generation of renderings. Secondly, during the generation process, two different texts are set for each sketch, and control experiments are carried out separately to avoid contingency and errors in the experimental process. The content of the control experiment includes the generation of renderings without adding the embedded rendering LoRA model, the generation of renderings when the redrawing amplitude is set to 0.4, and the generation of renderings when the redrawing amplitude is set to 0.5. Therefore, 6 renderings are output for the same sketch. The generated results are as follows Fig.12 shown.

[0259] The renderings generated by the designer's creative hand-drawn sketches have different presentation results at different redrawing amplitudes. When the redrawing amplitude is 0.4, the renderings have been given some real materials, but there is still room for improvement in details. When the redrawing amplitude is 0.5, the renderings are more realistic, more delicate, and richer in details. The renderings generated when the redrawing amplitude is 0.5 are selected as evaluation samples.

[0260] Step S4: Evaluation of multi-image product rendering schemes

[0261] First, the focus group discussion concluded that the rendering scheme generated when the redrawing amplitude was 0.5 was better than the redrawing amplitude of 0.4 and the embedded rendering LoRA model was not set. Then, the TOPSIS method was used to comprehensively evaluate the car rendering scheme generated when the redrawing amplitude was 0.5. The evaluation index was the five emotional images extracted above, namely "fashion", "smooth", "exquisite", "simple", and "texture". Combined with the weights of each emotional image calculated by the priority chart-entropy weight method, the final score and ranking of each rendering scheme can be obtained. The above 16 rendering schemes were made into a questionnaire with the five emotional images as evaluation indicators, and the subjects were invited to evaluate the degree of conformity of the 16 rendering schemes to the five emotional images. The questionnaire was scored using the Likert 5-point scale. Taking fashion as an example: when the subjects saw a car rendering, they scored it according to whether they thought it was fashionable. A total of 20 subjects' response data were collected, including 6 undergraduates and 14 postgraduates, all of whom were students majoring in industrial design.

[0262] The collected questionnaires were processed and the integrated data were analyzed using the TOPSIS method. The final evaluation results of the 16 samples are shown in Table 16 below.

[0263] Table 15. Positive and negative ideal solutions

[0264]

[0265]

[0266] Table 16. TOPSIS evaluation results

[0267]

[0268] like Fig.13 As shown in the evaluation results, sample 1 has the highest comprehensive score, so sample 1 is selected as the optimal solution. The designer uses the modeling software to complete the 3D engineering digital modeling of the optimal solution. The modeling results are shown in Fig.14 shown.

[0269] It should be understood that the above description of the preferred embodiment is relatively detailed and cannot be regarded as limiting the scope of patent protection of the present invention. Under the enlightenment of the present invention, ordinary technicians in this field can also make substitutions or modifications without departing from the scope of protection of the claims of the present invention, which all fall within the scope of protection of the present invention. The scope of protection requested for the present invention shall be based on the attached claims.

Claims

1. A method for generating product effect diagrams based on entropy weight priority diagram and stable diffusion model. It is characterized in that The following steps are involved: Step S1: Multi-image sentiment analysis and confirmation; Step S2: product sketch analysis and drawing; Step S3: generating a multi-image product rendering scheme; Step S4: Evaluation of multi-image product rendering schemes.

2. According to claim 1, the method for generating and designing multi-image product renderings based on entropy weight priority graph and stable diffusion model, It is characterized in that The specific contents of the multi-image sentiment analysis and confirmation in step S1 are as follows: Step S11: Constructing a perceptual dictionary Collect user comment texts through web crawlers, and after data cleaning, word segmentation and part-of-speech tagging, use Jieba word segmentation technology to tag the online comment text data, obtain all adjectives, remove invalid or similar adjectives, classify them, list the network relationships between adjectives, use TF-IDF technology to calculate the TF-IDF values ​​of the adjectives obtained and sort them from high to low according to the values, and select positive adjectives with high TF-IDF values ​​to construct a perceptual dictionary; Step S12: Initial weight calculation based on priority graph method Step S121: Determine the scoring criteria for each indicator Take the scores of u experts as rows and n evaluation indicators as columns, and calculate the average value S of each indicator. j : In the above formula: S j represents the average value of index j, j = 1, 2, ..., n; S ji represents the scoring result of expert i on the jth indicator, i = 1, 2, …, u; Step S122: Compare the average values Compare the two indicators, the larger one is scored as 1 point, the smaller one is scored as 0 point, and if they are completely equal, each is scored as 0.5 point, so as to obtain the judgment matrix A; where a ij is 0.5 or 0 or 1; Step S123: Calculate the weight p of each indicator j : In the above formula: p j is the weight of index j; a ij is the value of the i-th row and j-th column in the judgment matrix; Step S13: Multiple image rights confirmation based on entropy weight method Step S131: Data standardization The original data is standardized, and the evaluation index is set to the perceptual image vocabulary obtained above, the number of evaluation indexes is set to n, and the number of evaluation objects is set to m, where: j = 1, 2, 3 ..., n; i = 1, 2, 3 ..., m; Since all the indicators involved are positive indicators, we only need to use the positive indicator processing formula: In the above formula, Z ij is the positive index value of the jth indicator of the i-th evaluation object after processing, x ij It is the positive index value of the jth indicator of the ith evaluation object before processing. Step S132: Calculate information entropy and information effective value The information entropy value formula of the jth indicator is: In the above formula: is the proportion of the jth indicator of the i-th evaluation object to the total value of the indicator; ln∫ ij is the value of the jth indicator of the evaluation object i; The information effect value of the jth indicator is: d j =1-e j (6) Step S133: Calculate the weight of each indicator Calculate the indicator weights. The weight of the jth indicator is: The weights obtained by the two methods are normalized to obtain the specific multi-image weights, and the formula is as follows: In the above formula, p and q are the weights obtained by two methods.

3. According to claim 1, the method for generating and designing multi-image product renderings based on entropy weight priority diagram and stable diffusion model, It is characterized in that The specific contents of the product sketch analysis and drawing in step S2 are as follows: Firstly, the key styling elements that affect the overall styling image of the product are determined through focus group interviews, and the morphological analysis method is used to extract the styling elements from the selected product morphology, so as to construct an element sample set as the basis for fuzzy comprehensive evaluation; then, the user's multi-image emotion is used as the evaluation index of fuzzy comprehensive evaluation, and the fuzzy comprehensive evaluation method is used to evaluate the element sample set, and the element sample with the highest comprehensive score is used as the sketch image board, and the designer then draws the sketch based on the sketch image board; The specific process of the fuzzy comprehensive evaluation is as follows: Step S21: Determine the index set and weight vector The index set is I = {x 1 ,x 2 ,...x p }, that is, the user emotional image extracted in step S1; the weight vector is W = [w 1 ,w 2 ,...,w p ], the weight vector is a comprehensive weight obtained by the priority graph method and the entropy weight method; Step S22: Create a comment set V S comments constitute a comment set, denoted as V = {v 1 ,v 2 ,…v s }; Step S23: Establish a single indicator evaluation vector The evaluation matrix R is obtained by integrating the evaluation vectors of various indicators. ij ) p*s , where r ij is the evaluation result of the i-th indicator, and p is the number of indicators; Step S24: Synthesize the fuzzy comprehensive evaluation result vector Use appropriate operators to synthesize W and the evaluation matrix R to obtain the fuzzy comprehensive evaluation result vector A of the evaluated sample:

4. The method for generating and designing multi-image product renderings based on entropy weight priority graph and stable diffusion model according to claim 1, It is characterized in that The specific steps of generating the multi-image product rendering scheme in step S3 are: Step S31: Contour extraction The Canny model in ControlNet is used to extract the contour of the input sketch, and then the extracted contour is input into the rendering generation model, so as to accurately control the generation of the rendering; The network structure of the ControlNet consists of two parts, namely locked copy and trainable copy; the locked copy part fixes the original weights of the stable diffusion model to ensure that the image generation ability of the model remains unchanged during the subsequent training process, thereby providing a stable performance for the model; and the parameters of the trainable copy part are initialized to the parameters of the corresponding trainable copy, which will be updated and optimized in the subsequent training process of ControlNet, so that the entire network learning is adjusted to the task specified by the user; Step S32: Embedding LoRA model into stable diffusion model The stable diffusion model is used as the framework of the entire large model for effect image generation, and the LoRA model that maps the emotional image of the target user is embedded in it to form the large model for effect image generation. Step S33: training the large model for generating effect pictures to obtain the large model for generating effect pictures with target emotional tendency, inputting the hand-drawn sketches of designers into the trained large model for generating effect pictures to generate product effect picture solutions in batches; During the training of the stable diffusion model, noise is continuously added to the image to destroy the image. During the inference, the noisy image is denoised to restore the original image. During the T iterations of the training process, x at any time t Can be x 0 and β means that x at any time 0 It can also be represented by x t And the noise z is obtained: In the above formula, a and β represent the noise intensity; In the above formula, x 0 For the initial clean image, add noise z to generate the noisy image x t The second formula is derived from the first formula, which means that the noise image is subtracted from a certain intensity of noise to obtain the image x 0 ; The obtained noisy image x t Applied to the reverse diffusion stage of the model; In the reverse diffusion stage, the reverse process obeys the Gaussian distribution. Given the original clear image, the reverse process can be converted into a forward process through the Bayesian formula, and then the reverse process distribution can be calculated. The formula is expressed as: In the above formula, it refers to the given x 0 In this case, the reverse process obeys Gaussian distribution, and the Bayesian formula is used to transform the reverse process into a forward process; In the above formula, x t and x 0 can be converted to each other, and the mean can also be obtained from x t After subtracting different noises, the product rendering scheme can be obtained through the reverse diffusion stage and additional parameter control; In order to generate high-quality product renderings, the optimization of the model in the training phase includes: Fitting distribution The model estimation results follow a Gaussian distribution: In the above formula, the mean formula includes x t and noise, due to x t It is known during training that we only need to get the estimated noise of the model output, which can be obtained by the model using x t and t are estimated; in the reverse diffusion stage, the generated result of the previous step is input, the estimated noise of each step in the reverse diffusion stage is calculated, and then the noise is subtracted from the input to obtain the output of the current time step; Reverse Process Distribution Compare the inverse process distribution with the estimated distribution of the model. Since the KL divergence can be used to describe the difference between the two distributions and the KL divergence of the multivariate Gaussian distribution has a closed-form solution, it is deduced that the loss function becomes the KL divergence of the two Gaussian distributions, expressed as: L t =D KL (q(x t |x t+1 ,x 0 )||p θ (x t |x t+1 ));1≤t≤T-1 (15) In the above formula, q distribution is the inverse process distribution, p is the model estimated distribution, and the training loss is the KL divergence of the two Gaussian distributions, that is, the gap between the two distributions; Taking the above trained stable diffusion model as the basis, LoRA adapts to the downstream tasks by only training some new parameters according to the data of the corresponding downstream tasks. After the new parameters are trained, the new parameters are merged with the old model parameters by using the re-parameterization method, which can not only achieve the effect of fine-tuning the entire model on the new task, but also will not increase the inference time during inference.

5. The method for generating and designing multi-image product renderings based on entropy weight priority graph and stable diffusion model according to claim 1, It is characterized in that The specific content of the multi-image product rendering scheme evaluation in step S4 is: The TOPSIS method is used to evaluate the product rendering schemes with multi-image emotions as the evaluation index, combined with the weights obtained by the priority diagram-entropy weight method, and the scheme with the highest score is selected as the final scheme to ensure that the design best meets the user's emotional imagery; The calculation process of the TOPSIS method is as follows: Step S41: Calculate the comprehensive weight h according to the priority graph-entropy weight method j Establish the weighted matrix V with the standard matrix R, and determine the positive ideal solution (V + ) and the negative ideal solution (V - ): V=h j ×r ij (16) Step S42: Calculate the distance between each evaluation object and the positive ideal solution and the negative ideal solution: Step S43: Calculate the closeness C between the evaluation object and the ideal solution i : In the above formula: C i The larger the value is, the closer the i-th rendering is to the optimal level. i The value range is from 0 to 1; The TOPSIS method is used to screen out the product rendering scheme that best meets multiple images, and the designer conducts subsequent design activities based on the rendering scheme.

Images