Low-grade road aesthetics optimization method based on machine learning and fixed diffusion model

Abstract

The invention discloses a low-grade road aesthetics optimization method based on machine learning and a fixed diffusion model, and relates to the field of low-grade road aesthetics evaluation, and the method comprises the following steps: S1, data preparation: collecting low-grade road driving environment data, and carrying out the data processing; s2, an optimization object is determined based on aesthetic calculation, XGBoost and SHAP are combined to establish an aesthetic calculation model of the low-grade road environment, diversity, uniformity and symmetry of road environment elements are analyzed, an overall aesthetic score and an aesthetic feature score of the low-grade road environment are calculated, and the optimization object is determined; s3, performing intelligent optimization on the filtered optimization object by adopting a fixed diffusion model, wherein the fixed diffusion model comprises a diffusion model, a variational encoder VAE and a condition control module; and S4, verifying the optimization effect, and comparing the optimization scheme effect based on the fixed diffusion model with the optimization scheme effect based on the CycleGAN by adopting an aesthetic calculation model. According to the method, the automation and intelligence level in the aspect of road environment aesthetic quality optimization is improved.

Description

Technical Field

[0001] The present invention relates to the field of low-level road aesthetics evaluation, and in particular to a low-level road aesthetics optimization method based on machine learning and a fixed diffusion model. Background Art

[0002] In low-level road environments, drivers' aesthetic preferences are mainly influenced by road environment elements such as semantic, color, and texture information. These information stimulate different areas of the brain's visual cortex, leading to different aesthetic preferences. Drivers show different aesthetic preferences for different semantic information. Natural semantic information elements such as vegetation, water, and meadows are more popular. Richer color and texture information is positively correlated with drivers' aesthetic preferences for the road environment. The matching and distribution of colors not only enhance the visual appeal of the road environment, but also provide clear visual guidance for drivers to understand the road environment. Road environment elements are an important part of the aesthetic quality of the road environment and are the information source for computational analysis of road environment aesthetics.

[0003] Current road environment aesthetic computational models are usually implemented by extracting corresponding road environment elements. However, these models do not further explore the aesthetic characteristics of road environment elements and lack interpretability. In addition, existing studies on subjective aesthetic evaluation of road environments have found that aesthetic characteristics are closely related to the aesthetic quality of road environments, but these characteristics rely on empirical scoring and have not yet been objectively quantified.

[0004] The optimization method of road environment aesthetic quality relies on a large amount of experimental data and expert experience, and it is difficult to verify the optimization effect through intelligent generation. Although the optimization method of low-level road environment aesthetic quality based on human factors engineering can improve driving safety and comfort, it is costly because it requires a large amount of psychological and physiological data. The optimization method of urban and rural road environment landscape layout based on design specifications relies on expert experience, and the degree of automation of scheme modification is low.

[0005] Therefore, it is necessary to provide a low-level road aesthetic optimization method based on machine learning and fixed diffusion model to solve the above problems. Summary of the invention

[0006] The purpose of the present invention is to provide a method for optimizing the aesthetics of low-level roads based on machine learning and a fixed diffusion model, which objectively calculates the aesthetic quality of the low-level road environment based on the aesthetic characteristics of the road environment elements, and emphasizes the importance of improving its aesthetic quality. In addition, the automation and intelligence level in optimizing the aesthetic quality of the road environment is improved, and the repetitive manual work and reliance on expert experience in the road design process are reduced.

[0007] To achieve the above object, the present invention provides a low-level road aesthetic optimization method based on machine learning and a fixed diffusion model, comprising the following steps:

[0008] S1: Data preparation, collecting low-grade highway driving environment data and processing the data;

[0009] S2: Determine the optimization object based on aesthetic calculation, combine XGBoost and SHAP to establish an aesthetic calculation model for low-level road environment, analyze the diversity, unity and symmetry of road environment elements, calculate the overall aesthetic score and aesthetic feature score of low-level road environment, and determine the optimization object;

[0010] S3: A fixed diffusion model is used to intelligently optimize the filtered optimization object. The fixed diffusion model includes a diffusion model, a variational encoder VAE, and a conditional control module.

[0011] S4: Verify the optimization effect and use the aesthetic calculation model to compare the optimization scheme based on the fixed diffusion model with the optimization scheme based on CycleGAN.

[0012] Preferably, in step S1, a GARMIN GDR35 driving recorder is used to collect data, and 2000 clear and unobstructed driving environment images are obtained through data cleaning and screening, covering five types of landscapes: forests, grasslands, snow-capped mountains, wastelands and cliffs. Experimental participants score 1500 of the environmental images at six levels to evaluate the overall aesthetic score and aesthetic feature score of the road environment.

[0013] Preferably, in step S1, low-level road environment elements are quantified from semantic information, color information and texture information, and the process of acquiring semantic information is as follows:

[0014] S11: Perform semantic segmentation on the original driving environment image according to the constructed semantic segmentation network;

[0015] S12: The semantic segmentation network uses ResNet50 as the backbone network for sampling;

[0016] S13: access the FPN network to further fuse the features;

[0017] S14: The sampled features are integrated through the Bagging algorithm. The semantic information obtained after semantic segmentation includes roads, vegetation, protective facilities and sky;

[0018] The process of obtaining color information is as follows: convert the driving environment image into grayscale, and then map the grayscale value to the original color image to form a color heat map. The larger the grayscale value, the brighter the color in the heat map. The color information is represented by the three attributes of Hue, Saturation and Value in the HSV color space.

[0019] The Gabor filtering method is used to extract the texture information of the driving environment. The specific process is as follows:

[0020] The direction angle, spatial aspect ratio, standard deviation and frequency of the defined Gabor filter are used to generate a filter that matches the size of the environment image and is applied to the driving environment image. After filtering, a filter response image in a specific direction and scale is obtained.

[0021] Preferably, in step S2, the diversity of the driving environment is quantified by diversity, which includes semantic diversity, color diversity and texture diversity. The semantic diversity value is calculated by weighted summation of information entropy of various semantic information. The larger the value, the more complex and diverse the semantic information; the color diversity is quantified by RGB channels, and the larger the value, the richer the color; the texture diversity is calculated by statistically calculating the variance mean of all local area features of the texture information image. The larger the value, the more diverse the texture information. The calculation formula is as follows:

[0022]

[0023] Div col =σ rgyb +0.3μ rgyb

[0024]

[0025] Among them, Div sem represents the semantic diversity value; p(c i ) represents the number of all pixels of a certain semantics in the image; C represents the number of semantic categories; Div col Represents the color diversity value; r, g, b represent the values ​​of the three color channels respectively; rg represents the difference between the red channel and the green channel; yb represents half of the sum of the red and green channels minus the blue channel; μ rgyb represents the average value of the combined rg and yb channels; σ rgyb Indicates the standard deviation of the combined rg and yb channels; Div tex Represents the texture diversity value; μ i Represents the mean value of the local area; Var(μ i ) represents the variance of the mean of all local regions; Represents the variance of the local area; represents the variance of all local area variance values; H i Represents the entropy value of the local area; Var(H i ) represents the variance of all local region entropy values.

[0026] Preferably, in step S2, the uniformity of the driving environment is reflected in semantic uniformity, color uniformity and texture uniformity;

[0027] The semantic segmentation image is processed by the Canny edge detection algorithm to obtain the edge features of the semantic segmentation image, and the ratio of the number of edge pixels to the total number of pixels is used as the semantic unity value. The larger the semantic unity value, the more chaotic the semantics and the worse the unity.

[0028] The color uniformity value is obtained by matching the color uniformity map that best matches the color heat map. The color uniformity value is obtained by calculating the hue distance and saturation difference. The smaller the color uniformity value, the more uniform the color.

[0029] The texture uniformity value is quantified by the mean standard deviation of the pixel values ​​of the texture information image. The larger the texture uniformity value, the greater the difference in texture information. The calculation formula is as follows:

[0030]

[0031] Among them, Uni sem Represents the semantic uniformity value; M represents the number of rows of pixels in the image; N represents the number of columns of pixels in the image; Canny(i,j) represents the pixel (i,j) detected by the Canny edge detection method to represent the edge; Uni col represents the color uniformity value; H(p) and S(p) represent the hue value and saturation value of pixel p respectively; S t (α) represents the most consistent color uniformity map; t represents the map type; α represents the rotation angle of the map; hue distance Indicates the arc length on the color wheel (radians); Uni tex Represents the texture uniformity value; Z represents the number of local windows; Tex i Represents the texture information image corresponding to the i-th local window; std(Tex i ) represents the standard deviation of the pixel values ​​of the texture information image corresponding to the i-th local window.

[0032] Preferably, in step S2, the symmetry of the driving environment is reflected in the visual balance of semantic symmetry, color symmetry and texture information symmetry;

[0033] The semantic symmetry value is obtained by extracting the number of pixels of four main types of semantic information through k-means clustering, and calculating the difference in pixel proportions of these semantic information in the left and right parts of the semantic segmentation image. The larger the semantic symmetry value, the greater the difference in semantic information distribution and the weaker the symmetry;

[0034] The color symmetry value is obtained by accumulating the color pixel value difference between the left and right parts of the color heat map. The smaller the color symmetry value, the smaller the color difference of the symmetrical pixels, and the better the symmetry;

[0035] Texture symmetry is represented by the mean of the absolute difference between the left and right parts of the texture information image. The larger the texture symmetry value, the greater the difference between the left and right parts of the texture, and the worse the symmetry. The calculation formula is as follows:

[0036]

[0037] Among them, Sym sem represents the semantic symmetry value; C represents the number of semantic categories; Indicates the pixel ratio of the i-th semantic in the left half; Indicates the pixel ratio of the i-th semantic in the right half; Sym col Represents the color symmetry value; A represents the height of the image; B represents the width of the image; Col(P(i,j)) represents the color pixel value of the pixel point P(i,j) at position (i,j) on the color heat map; Col(P(Ai,j)) represents the color pixel value of the pixel point P(i,j) symmetrical to the center line of the color heat map; Sym tex Represents the texture symmetry value; Tex(P(i,j)) represents the pixel value of the pixel point P(i,j) at the position (i,j) of the texture information image; Tex(P(Ai,j)) represents the pixel value of the symmetric point P(Ai,j) of the pixel point P(i,j) about the center line of the texture information image.

[0038] Preferably, in step S2, the optimization method of the optimization object is as follows:

[0039] S21: XGBoost is used to predict the overall aesthetic score and aesthetic feature score of low-level road environments to determine the optimization object and optimization focus. SHAP is used to interpret the model output and analyze the specific impact of semantic diversity, color diversity, and texture diversity on diversity.

[0040] S22: In XGBoost, the decision tree is based on the optimal w tj Make a split and stop when the node depth reaches the maximum depth:

[0041]

[0042] In the formula, G tj and H tj are the first-order derivative and second-order derivative of the leaf nodes respectively; J is the number of leaf nodes; w tj is the optimal value of the jth leaf node of the tth decision tree, λ is a trade-off parameter, and γ represents the complexity of the leaf;

[0043] S23: The aesthetic calculation model is evaluated by the mean square error (MSE). The smaller the MSE, the smaller the difference between the calculated value and the observed value, and the better the model performance. The calculation formula is as follows:

[0044]

[0045] Where N is the number of samples; y i is the true value of the i-th sample; is the calculated value of the i-th sample;

[0046] S24: SHAP is used to explain the output of the aesthetic computational model. The calculation formula is as follows:

[0047]

[0048] Among them, φ i represents the contribution of factor i; N represents the set of all input factors; n represents the total number of samples of the factor; ν is the given model; S represents the set of all observed factors.

[0049] Preferably, in step S3, the process of intelligently optimizing the filtered optimization object by the fixed diffusion model is as follows:

[0050] S31: The input image x is converted to the latent space through the encoder ε to obtain the latent image z of the latent space;

[0051] S32: The diffusion model continuously adds noise to the potential image z and diffuses forward to obtain the final noise image z T ;

[0052] S33: The conditional control module further processes the input image x and text prompt as a constraint condition τ θ (y), and by making the noise predictor interpret, the desired image is obtained by controlling the reverse diffusion process;

[0053] S34: U-Net noise predictor for diffusion model on noise map z T De-noising is performed to obtain a noise-free latent image z, thus realizing the reverse diffusion process;

[0054] S35: The decoder converts the latent space output image z to the pixel space to obtain the final image generator

[0055] Preferably, in step S3, the loss function L of the fixed diffusion model is SDM It shows the expected value of all noise sample losses of the aesthetic learning model under certain image and condition information, L SDM The calculation formula is as follows:

[0056]

[0057] Where y represents the text prompt condition information; ε(x) represents the potential image mapped to the latent space after being compressed by the encoder ε; ∈~N(0,1) represents that the noise sample ∈ conforms to the standard normal distribution; E ε(x),y,∈～N(0,1),t Represents the expected value of the model's loss on all possible noise samples of the image that conform to the normal distribution; represents the square of the L2 norm of the difference between the predicted noise calculated by the noise predictor and the generated noise; t is the uniformly sampled time step; z t represents the potential representation obtained from the encoder ε; θ represents the parameters of the model; τ θ (y) represents the transformation of conditional information y through the conditional control module with parameter θ.

[0058] Therefore, the present invention adopts the above-mentioned low-level road aesthetic optimization method based on machine learning and fixed diffusion model, which has the following beneficial effects:

[0059] (1) The present invention generates corresponding text prompts according to the optimization object and inputs them into the fixed diffusion model to intelligently generate optimized low-level road environment images. The overall aesthetic score and aesthetic feature score are increased by 64.5% and 121.5% on average, respectively.

[0060] (2) The present invention calculates the overall aesthetic score and aesthetic feature score of the road environment through an aesthetic calculation model to determine the optimization object, providing an interpretable framework for road environment design, enabling designers to optimize the road environment in a more targeted manner.

[0061] (3) The present invention improves the automation and intelligence level in optimizing the aesthetic quality of the road environment, and reduces the repetitive manual work and reliance on expert experience in the road design process.

[0062] (4) The present invention links road environment elements such as semantics, color, and texture with aesthetic features to achieve objective quantitative analysis of aesthetic calculations. Compared with the popular image generation algorithm CycleGAN, the fixed diffusion model provides better optimization results.

[0063] (5) The road environment intelligent optimization technology based on aesthetic calculation proposed in the present invention realizes efficient solution generation, has significant optimization effect, improves the practicality of aesthetics, and is of great significance to the optimization, design and transformation of the road environment.

[0064] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 is a flow chart of a low-level road aesthetic optimization method based on machine learning and a fixed diffusion model of the present invention;

[0066] Figure 2 It is the overall framework diagram of the low-level road aesthetic optimization method based on machine learning and fixed diffusion model in the present invention;

[0067] Figure 3 It is a diagram of the semantic segmentation network structure in the present invention;

[0068] Figure 4 is a flow chart of a fixed diffusion model in the present invention;

[0069] Figure 5 is a fitting curve diagram of the observed values ​​and the calculated values ​​in the present invention;

[0070] Figure 6 It is a result diagram of the shape interpretation of each type of aesthetic feature score in the present invention. DETAILED DESCRIPTION

[0071] The technical solution of the present invention is further described below through the accompanying drawings and embodiments.

[0072] Unless otherwise defined, technical or scientific terms used in the present invention shall have the common meanings understood by one having ordinary skills in the field to which the present invention belongs.

[0073] The words "include" or "comprises" and the like used in the present invention mean that the elements before the word include the elements listed after the word, and do not exclude the possibility of also including other elements. The orientation or position relationship indicated by the terms "inside", "outside", "upper", "lower", etc. is based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation of the present invention. When the absolute position of the described object changes, the relative position relationship may also change accordingly. In the present invention, unless otherwise clearly specified and limited, the terms "attachment" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral body; it can be directly connected, or indirectly connected through an intermediate medium, and it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0074] Example

[0075] like Figure 1 As shown, the present invention provides a low-level road aesthetic optimization method based on machine learning and fixed diffusion model, including the following steps: the optimization method framework is as follows Figure 2 As shown;

[0076] S1: Data preparation, collecting low-grade highway driving environment data and processing the data;

[0077] The data was collected using a GARMIN GDR35 driving recorder. Through data cleaning and screening, 2000 clear and unobstructed driving environment images were obtained, covering five types of landscapes: forests, grasslands, snow-capped mountains, wastelands, and cliffs. The experimental participants scored 1500 of the environmental images at six levels to evaluate the overall aesthetic score and aesthetic feature score of the road environment. As shown in Table 1, the gender ratio of the experimental participants was balanced, with an age range of 23-50 years old, an average age of 30.6 years old, and a standard deviation of 6.1 years old.

[0078] Table 1

[0079]

[0080] In step S1, low-level road environment elements are quantified based on semantic information, color information, and texture information. The semantic segmentation network structure is as follows: Figure 3 As shown in Figure 2, the process of obtaining semantic information is as follows:

[0081] S11: Perform semantic segmentation on the original driving environment image according to the constructed semantic segmentation network;

[0082] S12: The semantic segmentation network uses ResNet50 as the backbone network for sampling;

[0083] S13: access the FPN network to further fuse the features;

[0084] S14: The sampled features are integrated through the Bagging algorithm. The semantic information obtained after semantic segmentation includes roads, vegetation, protective facilities and sky;

[0085] The process of obtaining color information is as follows: convert the driving environment image into grayscale, and then map the grayscale value to the original color image to form a color heat map. The larger the grayscale value, the brighter the color in the heat map. The color information is represented by the three attributes of Hue, Saturation and Value in the HSV color space.

[0086] The Gabor filtering method is used to extract the texture information of the driving environment. The specific process is as follows:

[0087] The direction angle, spatial aspect ratio, standard deviation and frequency of the defined Gabor filter are used to generate a filter that matches the size of the environment image and is applied to the driving environment image. After filtering, a filter response image in a specific direction and scale is obtained.

[0088] S2: Determine the optimization object based on aesthetic calculation, combine XGBoost and SHAP to establish an aesthetic calculation model for low-level road environment, analyze the diversity, unity and symmetry of road environment elements, calculate the overall aesthetic score and aesthetic feature score of low-level road environment, and determine the optimization object;

[0089] In step S2, the diversity of the driving environment is quantified by diversity, which includes semantic diversity, color diversity and texture diversity. The semantic diversity value is calculated by weighted summation of information entropy of various semantic information. The larger the value, the more complex and diverse the semantic information. The color diversity is quantified by RGB channels. The larger the value, the richer the color. The texture diversity is calculated by statistically calculating the variance mean of all local area features of the texture information image. The larger the value, the more diverse the texture information. The calculation formula is as follows:

[0090]

[0091] Div col =σ rgyb +0.3μ rgyb

[0092]

[0093] Among them, Div sem represents the semantic diversity value; p(c i ) represents the number of all pixels of a certain semantics in the image; C represents the number of semantic categories; Div col Represents the color diversity value; r, g, b represent the values ​​of the three color channels respectively; rg represents the difference between the red channel and the green channel; yb represents half of the sum of the red and green channels minus the blue channel; μ rgyb represents the average value of the combined rg and yb channels; σ rgyb Indicates the standard deviation of the combined rg and yb channels; Div tex Represents the texture diversity value; μ i Represents the mean value of the local area; Var(μ i ) represents the variance of the mean of all local regions; Represents the variance of the local area; represents the variance of all local area variance values; H i Represents the entropy value of the local area; Var(H i ) represents the variance of all local region entropy values.

[0094] In step S2, the uniformity of the driving environment is reflected in the uniformity of semantics, color and texture;

[0095] The semantic segmentation image is processed by the Canny edge detection algorithm to obtain the edge features of the semantic segmentation image, and the ratio of the number of edge pixels to the total number of pixels is used as the semantic unity value. The larger the semantic unity value, the more chaotic the semantics and the worse the unity.

[0096] The color uniformity value is obtained by matching the color uniformity map that best matches the color heat map. The color uniformity value is obtained by calculating the hue distance and saturation difference. The smaller the color uniformity value, the more uniform the color.

[0097] The texture uniformity value is quantified by the mean standard deviation of the pixel values ​​of the texture information image. The larger the texture uniformity value, the greater the difference in texture information. The calculation formula is as follows:

[0098]

[0099] Among them, Uni sem Represents the semantic uniformity value; M represents the number of rows of pixels in the image; N represents the number of columns of pixels in the image; Canny(i,j) represents the pixel (i,j) detected by the Canny edge detection method to represent the edge; Uni col represents the color uniformity value; H(p) and S(p) represent the hue value and saturation value of pixel p respectively; S t (α) represents the most consistent color uniformity map; t represents the map type; α represents the rotation angle of the map; hue distance Indicates the arc length on the color wheel (radians); Uni tex Represents the texture uniformity value; Z represents the number of local windows; Tex i Represents the texture information image corresponding to the i-th local window; std(Tex i ) represents the standard deviation of the pixel values ​​of the texture information image corresponding to the i-th local window.

[0100] In step S2, the symmetry of the driving environment is reflected in the visual balance of semantic symmetry, color symmetry and texture information symmetry;

[0101] The semantic symmetry value is obtained by extracting the number of pixels of four main types of semantic information through k-means clustering, and calculating the difference in pixel proportions of these semantic information in the left and right parts of the semantic segmentation image. The larger the semantic symmetry value, the greater the difference in semantic information distribution and the weaker the symmetry;

[0102] The color symmetry value is obtained by accumulating the color pixel value difference between the left and right parts of the color heat map. The smaller the color symmetry value, the smaller the color difference of the symmetrical pixels, and the better the symmetry;

[0103] Texture symmetry is represented by the mean of the absolute difference between the left and right parts of the texture information image. The larger the texture symmetry value, the greater the difference between the left and right parts of the texture, and the worse the symmetry. The calculation formula is as follows:

[0104]

[0105] Among them, Sym sem represents the semantic symmetry value; C represents the number of semantic categories; Indicates the pixel ratio of the i-th semantic in the left half; Indicates the pixel ratio of the i-th semantic in the right half; Sym col Represents the color symmetry value; A represents the height of the image; B represents the width of the image; Col(P(i,j)) represents the color pixel value of the pixel point P(i,j) at position (i,j) on the color heat map; Col(P(Ai,j)) represents the color pixel value of the pixel point P(i,j) symmetrical to the center line of the color heat map; Sym tex Represents the texture symmetry value; Tex(P(i,j)) represents the pixel value of the pixel point P(i,j) at the position (i,j) of the texture information image; Tex(P(Ai,j)) represents the pixel value of the symmetric point P(Ai,j) of the pixel point P(i,j) about the center line of the texture information image.

[0106] In step S2, the optimization method of the optimization object is as follows:

[0107] S21: XGBoost is used to predict the overall aesthetic score and aesthetic feature score of low-level road environments to determine the optimization object and optimization focus. SHAP is used to interpret the model output and analyze the specific impact of semantic diversity, color diversity, and texture diversity on diversity.

[0108] S22: The core of XGBoost is to minimize the objective function. By using the second-order Taylor expansion, the constant term is removed and the objective function is simplified. In XGBoost, the decision tree is based on the optimal w tj Make a split and stop when the node depth reaches the maximum depth:

[0109]

[0110] In the formula, G tj and H tj are the first-order derivative and second-order derivative of the leaf nodes respectively; J is the number of leaf nodes; w tj is the optimal value of the jth leaf node of the tth decision tree, λ is a trade-off parameter, and γ represents the complexity of the leaf;

[0111] S23: The aesthetic calculation model is evaluated by the mean square error (MSE). The smaller the MSE, the smaller the difference between the calculated value and the observed value, and the better the model performance. The calculation formula is as follows:

[0112]

[0113] Where N is the number of samples; y i is the true value of the i-th sample; is the calculated value of the i-th sample;

[0114] S24: SHAP is used to explain the output of the aesthetic computational model. The calculation formula is as follows:

[0115]

[0116] Among them, φ i represents the contribution of factor i; N represents the set of all input factors; n represents the total number of samples of the factor; ν is the given model; S represents the set of all observed factors.

[0117] S3: A fixed diffusion model is used to intelligently optimize the filtered optimization object. The fixed diffusion model includes a diffusion model, a variational encoder VAE, and a conditional control module; Figure 4 As shown, in step S3, the process of intelligently optimizing the filtered optimization object by the fixed diffusion model is as follows:

[0118] S31: The input image x is converted to the latent space through the encoder ε to obtain the latent image z of the latent space;

[0119] S32: The diffusion model continuously adds noise to the potential image z and diffuses forward to obtain the final noise image z T ;

[0120] S33: The conditional control module further processes the input image x and text prompt as a constraint condition τ θ (y), and by making the noise predictor interpret, the desired image is obtained by controlling the reverse diffusion process;

[0121] S34: U-Net noise predictor for diffusion model on noise map z T De-noising is performed to obtain a noise-free latent image z, thus realizing the reverse diffusion process;

[0122] S35: The decoder converts the latent space output image z to the pixel space to obtain the final image generator

[0123] In step S3, the loss function L of the diffusion model is fixed SDMIt shows the expected value of all noise sample losses of the aesthetic learning model under certain image and condition information, L SDM The calculation formula is as follows:

[0124]

[0125] Where y represents the text prompt condition information; ε(x) represents the potential image mapped to the latent space after being compressed by the encoder ε; ∈~N(0,1) represents that the noise sample ∈ conforms to the standard normal distribution; E ε(x),y,∈～N(0,1),t Represents the expected value of the model's loss on all possible noise samples of the image that conform to the normal distribution; represents the square of the L2 norm of the difference between the predicted noise calculated by the noise predictor and the generated noise; t is the uniformly sampled time step; z t represents the potential representation obtained from the encoder ε; θ represents the parameters of the model; τ θ (y) represents the transformation of conditional information y through the conditional control module with parameter θ.

[0126] S4: Verify the optimization effect and use the aesthetic calculation model to compare the optimization scheme based on the fixed diffusion model with the optimization scheme based on CycleGAN.

[0127] Performance of Aesthetic Computational Models

[0128] The 1500 samples were randomly divided into 80% for training and 20% for testing. After standardizing the input features, grid search and five-fold cross validation were used to optimize the parameters of the overall aesthetic score calculation model and the three aesthetic feature score calculation models, as shown in Table 2. The mean square errors of these models were 0.10, 0.06, 0.12, and 0.06, respectively. In addition, through the analysis of random sampling, the calculated values ​​of these models are basically consistent with the observed values, such as Figure 5 Therefore, the aesthetic computational model proposed in this scheme performs well.

[0129] Table 2

[0130]

[0131]

[0132] Road environment and aesthetic features that need to be optimized

[0133] According to the data distribution characteristics of the evaluation indicators of the labeled samples, the mean of the overall aesthetic score of the road environment and the mean of the aesthetic feature score are both less than 3 points, as shown in Table 3. In addition, this scheme defines 3 points as a situation with good aesthetic quality, as shown in Table 1. Therefore, among the unlabeled samples that have undergone aesthetic calculations, samples with an overall aesthetic score of the road environment less than 3 points are identified as road environments that need to be optimized. In addition, the aesthetic feature with the lowest score in the road environment is identified as the aesthetic feature that needs to be optimized.

[0134] Table 3

[0135]

[0136] The most influential road environment elements

[0137] The SHAP method is used to analyze the influencing factors of each aesthetic feature score from two aspects: global feature contribution and individual feature contribution. Figure 6 The global feature contribution and individual feature contribution are described as shown. Figure 6 (a) Represents the global feature contribution, illustrating the relative importance of the independent variables sorted by absolute SHAP values. Figure 6 (b) represents individual feature contribution, each point represents a sample, the color corresponds to the feature value, and the horizontal axis value of the point is the SHAP value. When the SHAP value of the black solid point is positive, it means that the independent variable with a larger feature value has a positive impact on the aesthetic feature score.

[0138] In terms of diversity score, color diversity is the most important variable, with the highest absolute SHAP value of 0.30. In the Unity score, color uniformity is the most important variable, with an absolute SHAP value of 0.25, which is much larger than texture uniformity and semantic uniformity. Finally, semantic symmetry is the feature that contributes the most to the symmetry score, with an absolute SHAP value of 0.39. The absolute SHAP values ​​of texture symmetry and color symmetry are both less than 0.08. Figure 6 (b) It can be seen that the black solid points of the diversity, unity and symmetry values ​​of various road environment elements (i.e., semantic, color and texture information) are mostly distributed on the positive axis of the SHAP value, while the white hollow points are distributed on the negative axis. This indicates that the aesthetic feature values ​​of these road environment elements are positively correlated with the scores of various aesthetic features.

[0139] Embodiment 1

[0140] When the road environment is a wasteland without vegetation, its overall aesthetic score is 1.9, indicating that there is still a lot of room for improvement in the aesthetic performance of the road environment. The diversity score, unity score, and symmetry score are 1.5, 3.1, and 2.4, respectively, indicating that the aesthetic feature that needs to be optimized is diversity. According to the results of SHAP, color diversity is the road environment element that has the greatest impact on the diversity score. Therefore, this embodiment mainly optimizes the aesthetic performance of the road environment by increasing color diversity.

[0141] Schemes 1 and 2 are optimization cases generated by the fixed diffusion model, and scheme 3 is a comparative optimization case generated by CycleGAN.

[0142] Solution 1 optimizes the road environment by adding grass. The text prompts include adding grass, rich colors, realistic, natural, and road environment. The number of iterations of the fixed diffusion model is 30. The overall aesthetic score of the optimized road environment is improved to 3.4 points, and the diversity score is improved to 3.6 points, which are 78.9% and 140.0% higher, respectively. This shows that the aesthetic performance has been significantly improved.

[0143] Trees also bring some optimization to the color of the road environment. Solution 2 increases the diversity of colors by adding trees. The text prompt word in Solution 1 is modified from "adding grass" to "adding green trees", and iterated 30 times. The overall aesthetic score of the optimized road environment is 3.7 points, and the road environment diversity score is 4.1 points, which are increased by 94.7% and 173.3% respectively.

[0144] For comparison, CycleGAN uses the road environment image with rich vegetation on the roadside as the target to generate Scheme 3. The overall aesthetic score of the road environment in Scheme 3 is 3.1 points, and the road environment diversity score is 3.3 points, both of which are lower than those of Optimized Case 1 and Optimized Case 2.

[0145] Therefore, the optimization cases generated based on the fixed diffusion model significantly improved the overall aesthetic score and diversity score of the road environment, and the improvement effect was better than that of the optimization cases generated based on CycleGAN.

[0146] Embodiment 2

[0147] In the original road environment, there are rocks without vegetation on the left and shrubs on the right. The overall aesthetic score of the road environment is 2.6, the diversity score is 3.6, the unity score is 2.1, and the symmetry score is 2.5. Therefore, the aesthetic feature that needs to be optimized in the original road environment is unity. Since color unity is the most important variable affecting the unity score, this embodiment considers adjusting the color unity performance to make the road environment more harmonious and unified.

[0148] Solution 1 is to increase the color uniformity by adding vegetation to cover the exposed rocks. The text prompts include adding vegetation on the left, similar colors to the vegetation on the right, realistic, natural, and road environment. After 30 iterations of the fixed diffusion model, the overall aesthetic score of the road environment increased to 4.1 and the uniformity score increased to 4.3, an increase of 57.7% and 104.8% respectively.

[0149] Scheme 2 improves the color uniformity by adding shrubs and flowers to cover the exposed rocks. Change "add vegetation on the left" in Scheme 1 to "add flowers on the left", and the number of iterations is the same as Scheme 1. The overall aesthetic score of the optimized road environment is 4.5, and the uniformity score is 4.1. The improvement effect exceeds 73.0%. Both optimization schemes generated based on the fixed diffusion model effectively optimize the aesthetic performance of the current road environment.

[0150] Scheme 3 is a comparative optimization case generated by CycleGAN. Its target is a road environment image where the roadside rocks are already richly covered with vegetation. The overall aesthetic score of the road environment is 3.3 and the uniformity score is 3.1. The improvement effect is less than 48.0%.

[0151] Therefore, the improvement effects of the two optimization cases generated based on the fixed diffusion model are better, and the effects of the optimization schemes are more realistic.

[0152] Embodiment 3

[0153] The overall aesthetic score of the road environment that needs to be optimized is 2.8, there are basically no trees on the left, and there are a few yellow trees on the right. The diversity score, unity score, and symmetry score are 3.5, 3.1, and 1.9, respectively. This shows that symmetry is a road aesthetic feature that needs to be optimized, and semantic symmetry is the most important factor affecting the symmetry score. Therefore, the main purpose of this embodiment is to improve the semantic symmetry of the road environment and make it more balanced and symmetrical.

[0154] Solution 1 enhances the symmetry of the road environment by adding semantic information such as vegetation on the left. The text prompts include adding many trees to cover the mountain on the left, adding more trees on the left similar to the right, symmetry and realism, and the number of iterations is 30. The overall aesthetic score of the optimized road environment increased by 50.0% to 4.2, and the symmetry score increased by 115.8% to 4.1.

[0155] Solution 2 optimizes the semantic information of vegetation on both sides of the road. The text prompts include adding trees of the same color and height on both sides of the road, and multiple trees covering the left and right hills, which is realistic. The number of iterations is the same as that of Solution 1. The overall aesthetic score of the optimized road environment is 3.7 points, and the symmetry score is 3.8 points, with improvement effects of 32.1% and 100.0% respectively.

[0156] The comparison scheme generated by CycleGAN in Scheme 3 takes the road environment image with rich vegetation on both sides of the road as the target, and its overall aesthetic score of the road environment is 3.2, and the road environment symmetry score is 2.8. The improvement effect of the two scores is 14.3% and 47.4% respectively, which is significantly smaller than that of Scheme 1 and Scheme 2. Therefore, both optimization schemes generated based on the fixed diffusion model can more effectively make the road environment more balanced and symmetrical by adjusting the semantic information.

[0157] Considering the display effect of the optimization scheme, scheme 3 generated based on CycleGAN is obviously less realistic and more random.

[0158] in conclusion

[0159] In three embodiments, the intelligent road environment optimization technology based on aesthetic calculation proposed in this scheme increased the overall aesthetic score and aesthetic feature score by an average of 64.5% and 121.5%, respectively. The optimization scheme generated based on CycleGAN increased the overall aesthetic score and aesthetic feature score by an average of 34.8% and 71.7%, respectively. Therefore, the optimization cases generated based on the fixed diffusion model can more effectively improve the aesthetic performance of the road environment. In addition, the fixed diffusion model can generate multiple more targeted optimization schemes under customized text prompts. In contrast, the optimization cases generated by CycleGAN based on the target image are often more random and lack authenticity.

[0160] Therefore, the optimization cases generated based on the fixed diffusion model can better meet the wishes of road environment designers and provide a convenient and intuitive demonstration method. Combined with the aesthetic calculation model, the effect of road environment optimization can be directly reflected.

[0161] The data source in this solution is static images of the road environment, which can be expanded to dynamic video data later.

[0162] Therefore, the present invention adopts the above-mentioned low-level road aesthetic optimization method based on machine learning and fixed diffusion model, and proposes a road environment intelligent optimization technology based on aesthetic calculation to achieve efficient solution generation, with significant optimization effect, improves the practicality of aesthetics, and is of great significance to the optimization, design and transformation of the road environment.

[0163] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A low-level road aesthetic optimization method based on machine learning and fixed diffusion model, characterized by: The following steps are involved: S1: Data preparation, collecting low-grade highway driving environment data and processing the data; S2: Determine the optimization object based on aesthetic calculation, combine XGBoost and SHAP to establish an aesthetic calculation model for low-level road environment, analyze the diversity, unity and symmetry of road environment elements, calculate the overall aesthetic score and aesthetic feature score of low-level road environment, and determine the optimization object; S3: A fixed diffusion model is used to intelligently optimize the filtered optimization object. The fixed diffusion model includes a diffusion model, a variational encoder VAE, and a conditional control module. S4: Verify the optimization effect and use the aesthetic calculation model to compare the optimization scheme based on the fixed diffusion model with the optimization scheme based on CycleGAN.

2. The low-level road aesthetic optimization method based on machine learning and fixed diffusion model according to claim 1, characterized in that: In step S1, a GARMIN GDR35 driving recorder was used to collect data. Through data cleaning and screening, 2000 clear and unobstructed driving environment images were obtained, covering five types of landscapes: forests, grasslands, snow-capped mountains, wastelands, and cliffs. The experimental participants scored 1500 of the environmental images at six levels to evaluate the overall aesthetic score and aesthetic feature score of the road environment.

3. The low-level road aesthetic optimization method based on machine learning and fixed diffusion model according to claim 1, characterized in that: In step S1, low-level road environment elements are quantified from semantic information, color information and texture information. The process of obtaining semantic information is as follows: S11: Perform semantic segmentation on the original driving environment image according to the constructed semantic segmentation network; S12: The semantic segmentation network uses ResNet50 as the backbone network for sampling; S13: access the FPN network to further fuse the features; S14: The sampled features are integrated through the Bagging algorithm. The semantic information obtained after semantic segmentation includes roads, vegetation, protective facilities and sky; The process of obtaining color information is as follows: convert the driving environment image into grayscale, and then map the grayscale value to the original color image to form a color heat map. The larger the grayscale value, the brighter the color in the heat map. The color information is represented by the three attributes of Hue, Saturation and Value in the HSV color space. The Gabor filtering method is used to extract the texture information of the driving environment. The specific process is as follows: The orientation angle, spatial aspect ratio, standard deviation and frequency of the Gabor filter are defined, and a filter matching the size of the environment image is generated and applied to the driving environment image. After filtering, a filter response image under a specific direction and scale is obtained.

4. The low-level road aesthetic optimization method based on machine learning and fixed diffusion model according to claim 1, characterized in that: In step S2, the diversity of the driving environment is quantified by diversity, and the diversity includes semantic diversity, color diversity and texture diversity; The semantic diversity value is calculated by weighted summation of information entropy of various semantic information. The larger the value, the more complex and diverse the semantic information. Color diversity is quantified by the RGB channels, with larger values ​​indicating richer colors; Texture diversity is calculated by counting the variance mean of all local area features of the texture information image. The larger the value, the more diverse the texture information. The calculation formula is as follows: Div. col =s rgyb +0.3m rgyb Among them, Div sem represents the semantic diversity value; p(c i ) represents the number of all pixels of a certain semantics in the image; C represents the number of semantic categories; Div col Represents the color diversity value; r, g, b represent the values ​​of the three color channels respectively; rg represents the difference between the red channel and the green channel; yb represents half of the sum of the red and green channels minus the blue channel; μ rgyb represents the average value of the combined rg and yb channels; σ rgyb Indicates the standard deviation of the combined rg and yb channels; Div tex Represents the texture diversity value; μ i Represents the mean value of the local area; Var(μ i ) represents the variance of the mean of all local regions; Represents the variance of the local area; represents the variance of all local area variance values; H i Represents the entropy value of the local area; Var(H i ) represents the variance of all local region entropy values.

5. The low-level road aesthetic optimization method based on machine learning and fixed diffusion model according to claim 1, characterized in that: In step S2, the uniformity of the driving environment is reflected in the uniformity of semantics, color and texture; The semantic segmentation image is processed by the Canny edge detection algorithm to obtain the edge features of the semantic segmentation image, and the ratio of the number of edge pixels to the total number of pixels is used as the semantic unity value. The larger the semantic unity value, the more chaotic the semantics and the worse the unity. The color uniformity value is obtained by matching the color uniformity map that best matches the color heat map. The color uniformity value is obtained by calculating the hue distance and saturation difference. The smaller the color uniformity value, the more uniform the color. The texture uniformity value is quantified by the mean standard deviation of the pixel values ​​of the texture information image. The larger the texture uniformity value, the greater the difference in texture information. The calculation formula is as follows: Among them, Uni sem Represents the semantic unity value; M represents the number of rows of pixels in the image; N represents the number of columns of pixels in the image; Canny(i,j) represents the pixel point (i,j) detected by the Canny edge detection method to represent the edge; Uni col represents the color uniformity value; H(p) and S(p) represent the hue value and saturation value of pixel p respectively; S t (α) represents the most consistent color uniformity map; t represents the map type; α represents the rotation angle of the map; hue distance Indicates the arc length on the color wheel (radians); Uni tex Represents the texture uniformity value; Z represents the number of local windows; Tex i Represents the texture information image corresponding to the i-th local window; std(Tex i ) represents the standard deviation of the pixel values ​​of the texture information image corresponding to the i-th local window.

6. The low-level road aesthetic optimization method based on machine learning and fixed diffusion model according to claim 1, characterized in that: In step S2, the symmetry of the driving environment is reflected in the visual balance of semantic symmetry, color symmetry and texture information symmetry; The semantic symmetry value is obtained by extracting the number of pixels of four main types of semantic information through k-means clustering, and calculating the difference in pixel proportions of these semantic information in the left and right parts of the semantic segmentation image. The larger the semantic symmetry value, the greater the difference in semantic information distribution and the weaker the symmetry; The color symmetry value is obtained by accumulating the color pixel value difference between the left and right parts of the color heat map. The smaller the color symmetry value, the smaller the color difference of the symmetrical pixels, and the better the symmetry; Texture symmetry is represented by the mean of the absolute difference between the left and right parts of the texture information image. The larger the texture symmetry value, the greater the difference between the left and right parts of the texture, and the worse the symmetry. The calculation formula is as follows: Among them, Sym sem represents the semantic symmetry value; C represents the number of semantic categories; Indicates the pixel ratio of the i-th semantic in the left half; Indicates the pixel ratio of the i-th semantic in the right half; Sym col Represents the color symmetry value; A represents the height of the image; B represents the width of the image; Col(P(i,j)) represents the color pixel value of the pixel point P(i,j) at position (i,j) on the color heat map; Col(P(Ai,j)) represents the color pixel value of the pixel point P(i,j) symmetrical to the center line of the color heat map; Sym tex Represents the texture symmetry value; Tex(P(i,j)) represents the pixel value of the pixel point P(i,j) at the position (i,j) of the texture information image; Tex(P(Ai,j)) represents the pixel value of the symmetric point P(Ai,j) of the pixel point P(i,j) about the center line of the texture information image.

7. The low-level road aesthetic optimization method based on machine learning and fixed diffusion model according to claim 1, characterized in that: In step S2, the optimization method of the optimization object is as follows: S21: XGBoost is used to predict the overall aesthetic score and aesthetic feature score of low-level road environments to determine the optimization object and optimization focus. SHAP is used to interpret the model output and analyze the specific impact of semantic diversity, color diversity, and texture diversity on diversity. S22: In XGBoost, the decision tree is based on the optimal w tj Make a split and stop when the node depth reaches the maximum depth: In the formula, G tj and H tj are the first-order derivative and second-order derivative of the leaf nodes respectively; J is the number of leaf nodes; w tj is the optimal value of the jth leaf node of the tth decision tree, λ is a trade-off parameter, and γ represents the complexity of the leaf; S23: The aesthetic calculation model is evaluated by the mean square error (MSE). The smaller the MSE, the smaller the difference between the calculated value and the observed value, and the better the model performance. The calculation formula is as follows: Where N is the number of samples; y i is the true value of the i-th sample; is the calculated value of the i-th sample; S24: SHAP is used to explain the output of the aesthetic computational model. The calculation formula is as follows: Among them, φ i represents the contribution of factor i; N represents the set of all input factors; n represents the total number of samples of the factor; ν is the given model; S represents the set of all observed factors.

8. The low-level road aesthetic optimization method based on machine learning and fixed diffusion model according to claim 1, characterized in that: In step S3, the process of intelligently optimizing the filtered optimization object by the fixed diffusion model is as follows: S31: The input image x is converted to the latent space through the encoder ε to obtain the latent image z of the latent space; S32: The diffusion model continuously adds noise to the potential image z and diffuses forward to obtain the final noise image z T ; S33: The conditional control module further processes the input image x and text prompt as a constraint condition τ θ (y), and make the noise predictor interpret it, and obtain the desired image by controlling the reverse diffusion process; S34: U-Net noise predictor for diffusion model on noise map z T De-noising is performed to obtain a noise-free latent image z, thus realizing the reverse diffusion process; S35: The decoder converts the latent space output image z to the pixel space to obtain the final image generator 9. The low-level road aesthetic optimization method based on machine learning and fixed diffusion model according to claim 1, characterized in that: In step S3, the loss function L of the diffusion model is fixed SDM It shows the expected value of all noise sample losses of the aesthetic learning model under certain image and condition information, L SDM The calculation formula is as follows: Where y represents the text prompt condition information; ε(x) represents the potential image mapped to the latent space after being compressed by the encoder ε; ∈~N(0,1) represents that the noise sample ∈ conforms to the standard normal distribution; E ε(x),y,∈～N(0,1),t Represents the expected value of the model's loss on all possible noise samples of the image that conform to the normal distribution; represents the square of the L2 norm of the difference between the predicted noise calculated by the noise predictor and the generated noise; t is the uniformly sampled time step; z t represents the potential representation obtained from the encoder ε; θ represents the parameters of the model; τ θ (y) represents the transformation of conditional information y through the conditional control module with parameter θ.

Images