Product aesthetic perception value difference evaluation method, system, equipment and medium

Abstract

The invention relates to the technical field of aesthetic evaluation, in particular to a product aesthetic perception value difference evaluation method, system and device and a medium. The evaluation method comprises the following steps: constructing a quality evaluation model, wherein the quality evaluation model comprises a feature extraction network, a fusion layer, a full connection layer and an output layer; the feature extraction network is used for extracting aesthetic features of the product appearance image; the fusion layer is used for fusion; the full connection layer is used for mapping the fusion features to different output layers; the output layer is used for outputting an evaluation result. According to the method, the aesthetic characteristics of the product appearance image are extracted and fused, and the quality evaluation model obtained by training in a multi-task learning mode is adopted to evaluate the aesthetic quality of the appearance image, so that the product aesthetic evaluation is more subjective while integrating objective aesthetic characteristics, and is easier to accept by the public.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of aesthetic evaluation, in particular to a product aesthetic perception value difference evaluation system, a product aesthetic perception value difference evaluation method, a product aesthetic perception value difference evaluation device and a product aesthetic perception value difference evaluation storage medium. BACKGROUND

[0002] Product aesthetics refers to the embodiment of aesthetic value in product design, which can affect people's perception and evaluation of products. When evaluating product aesthetics, the following differences can be considered: 1. Design concept: Different products may have different design concepts, some products emphasize simplicity and modernity, some products focus on antique style, and some products may pursue futurism. Consumers will evaluate whether the design concept of the product meets their preferences according to their aesthetic preferences. 2. Material and texture: The material and texture of the product also affect people's perception and evaluation. High-quality materials can enhance the aesthetic value of the product, while rough texture may affect the overall value of the product. 3. Color matching: Color matching plays a very important role in product aesthetics. Different color combinations can convey different emotions and atmosphere, and consumers will evaluate the aesthetic perception value of the product according to the coordination and uniqueness of color matching. 4. Combination of function and form: The functionality and form of the product need to be balanced in design. A product that is aesthetically appealing should have both practical functionality and unique form, and consumers will evaluate the value of the product according to this combination.

[0003] Existing product aesthetic evaluation methods often rely on the subjective evaluation of designers or public critics, making it difficult to form a unified aesthetic evaluation standard and meet the general cognition. In addition, some aesthetic evaluation methods based on neural networks only use the MOS score of subjective evaluation to guide the design objective method, and do not fully utilize the results of subjective evaluation. In fact, the results of subjective evaluation contain more information. SUMMARY

[0004] Therefore, it is necessary to propose a product aesthetic perception value difference evaluation method, system, device and medium to solve the problem that the subjective evaluation of personal consciousness and the objective evaluation of artificial intelligence are difficult to form an organic combination, resulting in a one-sided aesthetic evaluation and difficulty in obtaining universal recognition.

[0005] The present application is implemented by the following technical scheme: a product aesthetic perception value difference evaluation method includes the following steps:

[0006] The quality evaluation model includes a feature extraction network, a fusion layer, a full connection layer, and an output layer. The feature extraction network is used for aesthetic feature extraction of the product appearance image. The fusion layer is used for fusing the extracted aesthetic features into fusion features. The full connection layer is used for mapping the fusion features to different output layers. The output layer is used for outputting the evaluation result.

[0007] The evaluation process of the quality evaluation model is as follows: the quality evaluation model is trained by multi-task learning, and the model parameters that meet the test accuracy are retained. The product appearance image to be evaluated is input into the trained quality evaluation model to obtain multi-dimensional aesthetic evaluation.

[0008] The product aesthetic perception value difference evaluation method extracts the aesthetic features of the product appearance image through the feature extraction network, fuses multiple aesthetic features through the fusion layer, and maps them to multiple dimensional output layers. The quality evaluation model obtained by training in the multi-task learning manner can effectively evaluate the aesthetic quality of the appearance image, so that the product aesthetic evaluation is more subjective and more easily recognized by the public while comprehensively considering the objective aesthetic features.

[0009] Further, the feature extraction network selects a ResNet-50 feature extraction network. ResNet-50 includes 5 stages, namely: conv1, conv2, conv3, conv4, and conv5.

[0010] Wherein, the feature extraction of each stage is represented as:

[0011] f1=conv1(I;φ conv1 )

[0012] f2=conv2(f1;φ conv2 )

[0013] f3=conv3(f2;φ conv3 )

[0014] f4=conv4(f3;φ conv4 )

[0015] f5=conv2(f4;φ conv5 )

[0016] In the formula, I represents the input image, i.e. the product appearance image, f1, f2, f3, f4, and f5 represent the feature output results of the first layer, the second layer, the third layer, the fourth layer, and the fifth layer, respectively. They take the feature output of the previous layer as the input of the current layer. conv1(), conv2(), conv3(), conv4(), and conv5() represent the network structure of the first stage, the second stage, the third stage, the fourth stage, and the fifth stage of ResNet-50, respectively. φconvj denote the model parameters of the corresponding stage, j takes 1, 2, 3, 4, 5.

[0017] Further, the fusion layer reduces the high-dimensional features by global average pooling, and then fuses the reduced multiple features. The fusion method is horizontal splicing, and the formula is expressed as:

[0018] F(I; φ) = Ω[GP(f2), GP(f3), GP(f4), GP(f5)]

[0019] wherein, denote the feature representation of the input image I. I denotes the input image. denote all parameters of the entire feature extraction network. GP() denotes global average pooling. Ω() denotes splicing of multiple scales of features in the row direction into a row vector as the fused features.

[0020] Further, the output layer includes a quality regression model. The fused features are mapped to different output layers using a fully connected layer, and the corresponding aesthetic evaluation is obtained through the quality regression model, and different output layers share the same fused features. The output of the quality regression module is five evaluation levels.

[0021] Further, a SoftMax layer is added after the output of the fully connected layer to generate the probability of each evaluation level, and the process is described by the following formula:

[0022]

[0023] wherein, t = 1, 2, 3, 4. denote the distribution of different dimensions of aesthetic evaluation levels, and there are four output layers in total, which are image details, image color, image color coordination and overall aesthetic quality. fc t denote four different fully connected layer networks, and their network parameters are

[0024] The distribution of aesthetic evaluation levels is converted into a single MOS score for easy comparison with subjective analysis, and the conversion method is shown in the following formula:

[0025]

[0026] In the formula, μ denotes the mean of the aesthetic evaluation level of an output layer, σ 2 denotes the variance of the evaluation level, 1≤i≤5, s i denotes the five levels of aesthetic evaluation, and i takes a value from 1 to 5. denotes the probability corresponding to each level, and the probability value is between 0 and 1.

[0027] Further, the quality evaluation model is trained using a loss function based on EMD loss, which is expressed by the formula:

[0028]

[0029] wherein, represents the EMD distance between two distributions y, n represents the number of grades of scores. CDF y (k) represents the cumulative distribution function of y.

[0030] Further, in the multi-task learning training, an automatic weighted loss is used to balance the loss between multiple tasks, and the specific loss weighting method is as follows:

[0031]

[0032] In the formula, L(W) represents the overall loss of the entire task, L t (W) represents the loss of a certain task, here, t=4 represents that there are 4 tasks, which correspond to predicting four-dimensional scores respectively, and the four dimensions are image details, image color, image color coordination and overall aesthetic quality. The variable σ t is introduced to balance the loss of each task, and the regularization term is introduced to avoid excessive reduction of the weight of a certain task.

[0033] The application also provides a product aesthetic perception value difference evaluation system, which comprises a collection module, a preprocessing module and a quality evaluation model.

[0034] The collection module is used for collecting product appearance images. The preprocessing module is used for filtering the product appearance images. The quality evaluation model comprises a feature extraction network, a fusion module, a full connection layer, an output layer and a quality regression module, the feature extraction network is used for extracting aesthetic features of the product appearance images. The fusion module is used for inputting the extracted aesthetic features into a fusion layer to form fusion features. The full connection layer is used for mapping the fusion features to different output layers. The output layer comprises image details, image color, image color coordination and overall aesthetic quality. The quality regression model is used for outputting corresponding aesthetic evaluation. The output of the quality regression model is five evaluation grades.

[0035] The application also provides a product aesthetic perception value difference evaluation device, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, each functional module in the product aesthetic evaluation device is deployed in the manner of the product aesthetic perception value difference evaluation system, and the processor executes the computer program to realize the steps of the product aesthetic perception value difference evaluation method and realize feature extraction and evaluation of the product appearance images.

[0036] The application further provides a product aesthetic perception value difference evaluation storage medium, wherein a computer program is stored on the storage medium, and the program is used for executing the steps of the product aesthetic perception value difference evaluation method when executed by a processor.

[0037] Compared with the prior art, the application has the following beneficial effects:

[0038] The application extracts the aesthetic features of the product appearance image through the feature extraction network, fuses the multiple aesthetic features through the fusion layer, maps to the output layer of multiple dimensions, and is trained in a multi-task learning manner, so that the quality evaluation model obtained can effectively evaluate the aesthetic quality of the appearance image, so that the product aesthetic evaluation is more subjective and more easily recognized by the public while comprehensively evaluating the objective aesthetic features.

[0039] The application maps the fused aesthetic features to the aesthetic evaluation of multiple dimensions through the feature regression module, introduces the hyperparameter to balance the loss functions of multiple dimensions in the multi-task learning training, and the prediction tasks of each dimension output layer share the fused features, so that the quality evaluation model has strong aesthetic evaluation capability. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 FIG. 1 is a schematic diagram of the framework structure of the quality evaluation model in Embodiment 1 of the application. DETAILED DESCRIPTION

[0041] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the application.

[0042] It should be noted that when a component is referred to as being “mounted on” another component, it can be directly on the other component or there can be a middle component. When a component is referred to as being “disposed on” another component, it can be directly disposed on the other component or there can be a middle component. When a component is referred to as being “fixed on” another component, it can be directly fixed on the other component or there can be a middle component.

[0043] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description herein is for describing particular embodiments only and is not intended to be limiting of the application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0044] Embodiment 1: Please refer to Figure 1 The embodiment provides a product aesthetic perception value difference evaluation method for extracting aesthetic features of product appearance images and generating corresponding evaluations according to an evaluation model. The evaluation method comprises the following steps:

[0045] A quality evaluation model is constructed, which comprises a feature extraction network, a fusion layer, a full connection layer, and an output layer.

[0046] The feature extraction network is used for aesthetic feature extraction of the product appearance image. In this embodiment, ResNet is selected as the feature extraction network, and specifically, a ResNet-50 feature extraction network with a depth of 50 layers is selected. ResNet-50 comprises five stages, namely, conv1, conv2, conv3, conv4, and conv5.

[0047] In each stage, the feature extraction is represented as:

[0048] f1 = conv1 (I; φ conv1 )

[0049] f2 = conv2 (f1; φ conv2 )

[0050] f3 = conv3 (f2; φ conv3 )

[0051] f4 = conv4 (f3; φ conv4 )

[0052] f5 = conv2 (f4; φ conv5 )

[0053] In the formula, I represents an input image, i.e., a product appearance image, f1, f2, f3, f4, and f5 represent the feature output results of the first layer, the second layer, the third layer, the fourth layer, and the fifth layer respectively, which take the feature output of the previous layer as the input of the current layer, and conv1(), conv2(), conv3(), conv4(), and conv5() represent the network structures of the first stage, the second stage, the third stage, the fourth stage, and the fifth stage of ResNet-50 respectively. φ convjmodel parameters representing the corresponding stage, j takes 1, 2, 3, 4, 5.

[0054] The fusion layer is used to fuse the extracted aesthetic features into a fusion feature. The fusion layer reduces the high-dimensional feature by global average pooling, and then fuses the multiple features after dimension reduction. The fusion method is horizontal splicing, and the formula is expressed as:

[0055] F(I; φ) = Ω[GP(f2), GP(f3), GP(f4), GP(f5)]

[0056] Wherein, represents the feature representation of the input image I, I represents the input image, represents all the parameters of the entire feature extraction network, GP() represents global average pooling, Ω() represents splicing of multiple scales of features along the row direction, and the spliced row vector is used as the fused feature.

[0057] The fully connected layer is used to map the fused feature to different output layers. After the output of the fully connected layer, a SoftMax layer is added to generate the probability of each evaluation level, and the process is described by the following formula:

[0058]

[0059] Wherein, t = 1, 2, 3, 4, represents the distribution of different dimensions of aesthetic evaluation levels, and there are four output layers in total, which are image details, image color, image color coordination and overall aesthetic quality, fc t represents 4 different fully connected layer networks, and their network parameters are respectively

[0060] The distribution of aesthetic evaluation levels is converted into a single MOS score for easy comparison with subjective analysis, and the conversion method is shown in the following formula:

[0061]

[0062] In the formula, μ represents the mean of the aesthetic evaluation level of each output layer, σ 2 represents the variance of the evaluation level, 1≤i≤5, s i i represents the 5 levels of aesthetic evaluation, and i takes a value from 1 to 5. represents the probability corresponding to each level, and the probability value is between 0 and 1.

[0063] The output layer is used to output the evaluation result. The output layer comprises a quality regression model. The fused features are mapped to different output layers using a fully connected layer, and the corresponding aesthetic evaluation is obtained through the quality regression model, and different output layers share the same fused features. The output of the quality regression module is 5 evaluation levels.

[0064] The quality evaluation model is trained using a loss function based on EMD loss, which is expressed by the formula:

[0065]

[0066] wherein, represents the EMD distance between two distributions y, n represents the number of score levels. CDF y (k) represents the cumulative distribution function of y.

[0067] The evaluation process of the quality evaluation model is as follows: the quality evaluation model is trained by multi-task learning, and the model parameters that meet the test accuracy are reserved. In the multi-task learning training, an automatic weighted loss is used to balance the loss between multiple tasks, and the specific loss weighting method is as follows:

[0068]

[0069] In the formula, L(W) represents the overall loss of the entire task, and L t (W) represents the loss of a certain task. Here, t=4 represents that there are 4 tasks, which correspond to predicting four-dimensional scores respectively, and the four dimensions are image details, image color, image color coordination and overall aesthetic quality. A learnable variable σ t is introduced to balance the loss of each task, and a regularization term is introduced to avoid excessive reduction of the weight of a certain task.

[0070] The product appearance image to be evaluated is input into the trained quality evaluation model to obtain multi-dimensional aesthetic evaluation.

[0071] The product aesthetic perception value difference evaluation method of the embodiment extracts the aesthetic features of the product appearance image through the feature extraction network, fuses multiple aesthetic features through the fusion layer, and maps them to multiple dimensional output layers. Through multi-task learning, the obtained quality evaluation model can effectively evaluate the aesthetic quality of the appearance image, so that the product aesthetic evaluation is more subjective and more easily recognized by the public while comprehensively considering the objective aesthetic features. The magnet tile adopts a feature regression module to map the fused aesthetic features to multiple dimensional aesthetic evaluations. In the multi-task learning training, a hyperparameter is introduced to balance the loss functions of multiple dimensions, and the prediction tasks of each dimensional output layer share the fused features, so that the quality evaluation model has strong aesthetic evaluation capability.

[0072] Embodiment 2: The embodiment provides a product aesthetic perception value difference evaluation system, which can be controlled by the product aesthetic perception value difference evaluation method of embodiment 1. The system comprises a collection module, a preprocessing module, and a quality evaluation model.

[0073] The collection module is configured to collect product appearance images. The collection module can be a camera device, such as a video camera, a camera, or the like, which photographs or videos the product to be evaluated, thereby obtaining a plurality of product appearance images. The product appearance images are images taken from multiple perspectives to show the appearance of the product from different angles.

[0074] The preprocessing module is configured to perform filtering processing on the product appearance images. In some embodiments, the preprocessing module can also perform stretching, cropping, rotating, or the like on the product images to make the product images more consistent with general observation angles. The filtering processing can employ Gaussian filtering. In other embodiments, the product without color influence can also be subjected to grayscale processing.

[0075] The quality evaluation model comprises a feature extraction network, a fusion module, a fully connected layer, an output layer, and a quality regression module. The feature extraction network is configured to extract aesthetic features of the product appearance images. The fusion module is configured to input the extracted aesthetic features into a fusion layer to form fused features. The fully connected layer is configured to map the fused features to different output layers. The output layer comprises image details, image color, image color coordination, and overall aesthetic quality. The quality regression model is configured to output corresponding aesthetic evaluations. The output of the quality regression model is five evaluation levels. The specific functions and processing procedures of the quality evaluation model are described in embodiment 1 and will not be repeated here.

[0076] Embodiment 3: The embodiment provides a product aesthetic perception value difference evaluation device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor. The functional modules in the product aesthetic evaluation device are deployed in the manner of a product aesthetic perception value difference evaluation system according to the embodiment. When the processor executes the computer program, the steps of the product aesthetic perception value difference evaluation method according to embodiment 1 are implemented, and the feature extraction and evaluation of the product appearance images are implemented.

[0077] The device can be a smartphone, a tablet computer, a notebook computer, a desktop computer, a rack server, a blade server, a tower server, or a cabinet server (including a standalone server or a server cluster composed of multiple servers).

[0078] The memory (i.e., a readable storage medium) in the embodiment includes a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., an SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory can be an internal storage unit of the computer device, such as a hard disk or a memory of the computer device. In other embodiments, the memory can also be an external storage device of the computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device. Of course, the memory can also include both the internal storage unit and the external storage device of the computer device. In the embodiment, the memory is generally used to store an operating system and various application software installed on the computer device, etc. In addition, the memory can also be used to temporarily store various data that have been output or will be output.

[0079] The processor in some embodiments can be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor is generally used to control the overall operation of the computer device. In the embodiment, the processor is used to run the program code or process the data stored in the memory, thereby implementing the steps of the product aesthetic perception value difference evaluation method described above, thereby implementing the feature extraction and evaluation of the product appearance image and improving the precision and stability of the product aesthetic evaluation.

[0080] Embodiment 4: The embodiment provides a product aesthetic perception value difference evaluation storage medium, and the storage medium stores a computer program. The program is executed by a processor to perform the steps of the product aesthetic perception value difference evaluation method of embodiment 1.

[0081] Each technical feature of the above-described embodiments can be combined arbitrarily. To make the description concise, all possible combinations of the technical features in the above-described embodiments are not described, but it should be considered that any combination of the technical features is within the scope of the present disclosure, as long as the combination does not cause contradiction.

[0082] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be noted that for ordinary skilled persons in the art, without departing from the concept of the present application, several modifications and improvements can be made, which are within the scope of protection of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. A method for evaluating differences in perceived aesthetic value of products, which extracts aesthetic features from product appearance images and then generates corresponding evaluations based on an evaluation model; characterized in that, Includes the following steps: A quality evaluation model is constructed, comprising a feature extraction network, a fusion layer, a fully connected layer, and an output layer. The feature extraction network is used to extract aesthetic features from product appearance images. The fusion layer is used to fuse the extracted aesthetic features into a fused feature. The fully connected layer is used to map the fused feature to different output layers. The output layer is used to output the evaluation result. The evaluation process of the quality evaluation model is as follows: the quality evaluation model is trained through multi-task learning, and the model parameters that meet the test accuracy are retained; the appearance image of the product to be evaluated is input into the trained quality evaluation model to obtain a multi-dimensional aesthetic evaluation.

2. The method for evaluating differences in perceived aesthetic value of a product according to claim 1, characterized in that, The feature extraction network selected is the ResNet-50 feature extraction network; the ResNet-50 feature extraction network includes 5 stages, which are used to extract aesthetic features of different dimensions.

3. The method for evaluating differences in perceived aesthetic value of a product according to claim 1, characterized in that, The fusion layer reduces the dimensionality of high-dimensional features through global average pooling, and then fuses the multiple reduced features; wherein, the fusion method is horizontal concatenation, expressed by the formula: F(I;φ)=Ω[GP(f2), GP(f3), GP(f4), GP(f5)] in, The feature representation of the input image I; I represents the input image; This represents all parameters of the entire feature extraction network; GP() represents global average pooling; Ω() represents concatenating features at multiple scales along the row direction into a single row vector as the fused feature.

4. The method for evaluating differences in perceived aesthetic value of a product according to claim 1, characterized in that, The output layer contains a quality regression model; a fully connected layer is used to map the fusion features to different output layers, and the corresponding aesthetic evaluation is obtained through the quality regression model, and different output layers share the same fusion features; the output of the quality regression module is 5 evaluation levels.

5. The method for evaluating differences in perceived aesthetic value of a product according to claim 1, characterized in that, A SoftMax layer is added after the output of the fully connected layer to generate the probability of each rating level. The process is described by the following formula: Where t = 1, 2, 3, 4; This represents the distribution of aesthetic evaluation levels across different dimensions, with a total of four output layers: image detail, image color, image color harmony, and overall aesthetic quality; fc t This represents four different fully connected layer networks, and their network parameters are as follows:

6. The method for evaluating differences in perceived aesthetic value of a product according to claim 5, characterized in that, The quality assessment model is trained using a loss function based on EMD loss, expressed as follows: in, Let y represent two distributions. EMD distance between; n represents the number of grades; CDF y (k) represents the cumulative distribution function of y.

7. The method for evaluating differences in perceived aesthetic value of a product according to claim 5, characterized in that, In the multi-task learning and training, an automatic weighted loss is used to balance the losses among multiple tasks. The specific loss weighting method is as follows: In the formula, L(W) represents the total loss of the entire task, L t (W) represents the loss for a specific task. Here, t=4 indicates that there are four tasks, each corresponding to a score prediction across four dimensions: image detail, image color, image color harmony, and overall aesthetic quality. A learnable variable σ is introduced... t This is used to balance the loss for each task, and a regularization term is introduced. This is to prevent the weight of a certain task from being reduced excessively.

8. A product aesthetic perceived value difference evaluation system, which adopts the product aesthetic evaluation method as described in any one of claims 1 to 7, characterized in that, The aesthetic evaluation system includes: The data acquisition module is used to acquire images of the product's appearance. The preprocessing module is used to filter the product appearance image; The quality evaluation model includes a feature extraction network, a fusion module, a fully connected layer, an output layer, and a quality regression module. The feature extraction network is used to extract aesthetic features from the product appearance image. The fusion module is used to input the extracted aesthetic features into the fusion layer for fusion to form the fused features. The fully connected layer is used to map the fused features to different output layers. The output layers include image details, image color, image color harmony, and overall aesthetic quality. The quality regression model is used to output the corresponding aesthetic evaluation. The output of the quality regression model has 5 evaluation levels.

9. A device for evaluating differences in perceived aesthetic value of a product, comprising: The device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that each functional module in the product aesthetic evaluation device is deployed in the manner described in claim 8 as a product aesthetic perceived value difference evaluation system, and when the processor executes the computer program, it implements the steps of a product aesthetic perceived value difference evaluation method as described in any one of claims 1 to 7, thereby realizing feature extraction and evaluation of product appearance images.

10. A product aesthetic perceived value difference evaluation storage medium, wherein a computer program is stored thereon, characterized in that, When executed by a processor, the program performs the steps of a product aesthetic perceived value difference evaluation method as described in any one of claims 1 to 7.

Images