Adaptive image blur processing system and method based on deep convolutional neural network

Abstract

The present invention relates to the technical field of image processing, and discloses an adaptive image blur processing system and method based on a deep convolutional neural network, comprising: using a pre-trained semantic segmentation model to segment a foreground area and a background area in a landscape image; inputting the landscape image into a pre-trained visual attention model to distinguish target pixels in the landscape image, and marking a pixel area formed by connecting the target pixels as a significant attention area; calculating the overlap between the significant attention area and the foreground area, and when the overlap is less than or equal to a preset overlap threshold, using a pre-configured genetic optimization algorithm to obtain an optimal blur radius of the background area; performing Gaussian blur blur processing on the background area according to the optimal blur radius to obtain a blurred landscape image; the present invention is conducive to avoiding excessive or inappropriate blur processing of the landscape image.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and more specifically, to an adaptive image blur processing system and method based on a deep convolutional neural network. Background Art

[0002] In the digital age, competition among travel platforms is becoming increasingly fierce, and users' expectations for image quality are also constantly increasing. High-quality images can not only attract users' attention, but also effectively improve their overall experience and purchasing decisions. Therefore, how to improve the visual effects of uploaded images has become an urgent problem that travel platforms need to solve.

[0003] Image blur processing is an important image editing technology that can effectively enhance the beauty and professionalism of an image. By blurring the background, the main object in the image can be highlighted, interference can be reduced, and visual focus can be enhanced. Although existing image blur processing technologies have been applied in some fields, existing methods still have many shortcomings for the special needs of tourism platforms. For example, the blur processing judgment and blur processing process of existing tourism platforms rely on manual implementation, resulting in low blur processing efficiency and difficulty in effectively responding to the real-time processing needs of a large number of pictures on tourism platforms. In addition, for pictures of tourist attractions, existing blur technologies cannot perform personalized optimization processing according to different scenes (such as natural scenery, architecture, cultural landscapes, etc.), and excessive or inappropriate blur processing causes the landscape images to lose their natural feel and cannot guarantee the visual effects of the uploaded landscape images, thereby reducing the overall experience of the tourism platform. Summary of the invention

[0004] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides an adaptive image blur processing system and method based on a deep convolutional neural network.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] An adaptive image blur processing system based on a deep convolutional neural network, the system comprising:

[0007] The first segmentation module is used to obtain the landscape image uploaded to the platform and segment the foreground area and background area of ​​the landscape image using a pre-trained semantic segmentation model;

[0008] The second segmentation module is used to input the landscape image into the pre-trained visual attention model to distinguish the target pixels in the landscape image and mark the pixel area formed by connecting the target pixels as a salient attention area;

[0009] An overlap judgment module is used to calculate the overlap between the salient attention area and the foreground area, and when the overlap is less than or equal to a preset overlap threshold, obtain an optimal blur radius of the background area using a preconfigured genetic optimization algorithm;

[0010] The blur processing module is used to perform Gaussian blur blur processing on the background area according to the optimal blur radius to obtain a blurred landscape image.

[0011] Furthermore, segmenting the foreground area and the background area in the landscape image includes:

[0012] The edge detection algorithm is used to identify multiple object contours in the landscape image, and the pixel area surrounded by each object contour is used as the verification area;

[0013] Input each area to be verified into the pre-trained semantic segmentation model to obtain the category to which each area to be verified belongs;

[0014] Wherein, the categories include foreground categories and background categories;

[0015] The training logic of the semantic segmentation model is as follows:

[0016] Acquire historical semantic segmentation training data, and divide the historical semantic segmentation training data into a semantic segmentation training set and a semantic segmentation test set; the historical semantic segmentation training data includes a plurality of regions to be verified in the form of images and their corresponding category labels;

[0017] Constructing a first classifier, taking the region to be verified in the form of an image in the semantic segmentation training set as the input of the first classifier, and taking the category label in the semantic segmentation training set as the output of the first classifier, training the first classifier to obtain an initial semantic segmentation network;

[0018] Using the semantic segmentation test set to verify the model of the initial semantic segmentation network, outputting an initial semantic segmentation network that is greater than or equal to a preset first test accuracy threshold as a trained semantic segmentation model;

[0019] The corresponding area to be verified is divided into a foreground area or a background area according to its category.

[0020] Furthermore, the training logic of the visual attention model is as follows:

[0021] Acquire historical visual attention training data, and divide the historical visual attention training data into a visual attention training set and a visual attention test set; the historical visual attention training data includes a plurality of landscape images and a classification label for each pixel therein;

[0022] The classification labels include "0" and "1", "0" represents that the corresponding pixel is a non-target pixel, and "1" represents that the corresponding pixel is a target pixel;

[0023] Constructing a second classifier, taking the landscape image in the visual attention training set as the input of the second classifier, and taking the classification label of each pixel in the visual attention training set as the output of the second classifier, training the second classifier to obtain an initial visual attention network;

[0024] The initial visual attention network is model verified using the visual attention test set, and an initial visual attention network that is greater than or equal to the second test accuracy threshold is output as the trained visual attention model.

[0025] Furthermore, the generation logic of the classification label is as follows:

[0026] Get the attention intensity of each pixel in the landscape image;

[0027] Compare the attention-attracting intensity of each pixel with a preset attention-attracting intensity threshold;

[0028] If the attention-attracting intensity is greater than the attention-attracting intensity threshold, the corresponding pixel point is marked as a target pixel point;

[0029] If the attention-attracting intensity is less than or equal to the attention-attracting intensity threshold, the corresponding pixel point is marked as a non-target pixel point.

[0030] Furthermore, the process of obtaining the intensity of attracting attention is as follows:

[0031] Get the RGB value and brightness value of each pixel in the landscape image;

[0032] Substitute the RGB value and brightness value into the preset intensity calculation model to obtain the attention-attracting intensity of each pixel;

[0033] The expression of the strength calculation model is as follows:

[0034] ;

[0035] Where: Represents pixel The intensity of attracting attention, Represents pixel Color contrast, Represents pixel The brightness contrast ratio, and is a weight coefficient greater than zero, ;in, The calculation formula is: , The calculation formula is: ; Where: , G and B represent pixel points respectively Color values ​​in the red, green, and blue channels, , and Represented as pixels The color mean of the red, green, and blue channels in the neighborhood, , and Represented as pixels Luminance values ​​in the red, green, and blue channels, Represented as pixels The mean brightness of the red, green, and blue channels in the neighborhood.

[0036] Furthermore, the calculation of the overlap between the salient attention area and the foreground area includes:

[0037] Acquire a pixel point set of a salient attention area, and mark the pixel point set of the salient attention area as a first pixel point set;

[0038] Acquire a pixel point set of the foreground area, and mark the pixel point set of the foreground area as a second pixel point set;

[0039] Calculate the intersection of the first pixel point set and the second pixel point set, and calculate the union of the first pixel point set and the second pixel point set;

[0040] The ratio of the intersection to the union is marked as the overlap between the salient attention region and the foreground region.

[0041] Furthermore, when the overlap is greater than a preset overlap threshold, it is determined that the landscape image meets the set output standard, and the landscape image is directly displayed on the platform.

[0042] Furthermore, the method of obtaining the optimal blur radius of the background area by using a preconfigured genetic optimization algorithm includes:

[0043] a1: Initialize the population: randomly generate an original population, which contains K chromosomes, each chromosome represents a fuzzy radius, and K is an integer greater than zero;

[0044] a2: Fitness evaluation: Under each chromosome, obtain the average gradient change rate of the landscape image; input the average gradient change rate into the pre-built fitness function to calculate the fitness of each chromosome;

[0045] The logic for obtaining the average gradient change rate is as follows:

[0046] Extract the edge of the background area in the landscape image and calculate the gradient of each pixel in the edge in the horizontal and vertical directions;

[0047] Determine the gradient strength of each pixel in the edge portion according to the gradients in the horizontal and vertical directions;

[0048] Obtain the adjacent pixels of each pixel in the edge part, as well as the gradient strength of the adjacent pixels;

[0049] The average gradient change rate of the edge part is calculated based on the gradient strength of each pixel and its adjacent pixels;

[0050] The calculation formula of the average gradient change rate is as follows:

[0051] ;

[0052] Where: Represents the gradient change rate at the pixel point (x, y), Represents the gradient strength of the pixel point (x, y) in the edge part, Represents the gradient strength at the adjacent pixel point (x+1,y), Represents the gradient strength at the adjacent pixel point (x, y+1); is the average gradient change rate, N is the number of pixels in the edge part, is the pixel set of the edge part;

[0053] The calculation formula of the pre-constructed fitness function is: ; Where: For fitness, is the average gradient change rate, e is a natural constant;

[0054] a3: Selection: Use the roulette wheel method to select two chromosomes with high fitness in the original population as the father and mother;

[0055] a4: Crossover: Perform crossover operation on the father and mother to generate new chromosomes;

[0056] a5: Mutation: Perform mutation operation on the new chromosomes to obtain Y new chromosomes, combine the Y new chromosomes into a new population, replace the original population with the new population, and return to step a2;

[0057] a6: Repeat steps a2 to a5 above until the fitness of the chromosome in the original population or the new population is greater than or equal to the preset fitness threshold, or the number of iterations is greater than or equal to the preset maximum number of iterations threshold, and output the fuzzy radius represented by the corresponding chromosome as the optimal fuzzy radius.

[0058] An adaptive image blur processing method based on a deep convolutional neural network, the method comprising:

[0059] Obtain the landscape image uploaded to the platform, and use the pre-trained semantic segmentation model to segment the foreground and background areas in the landscape image;

[0060] Input the landscape image into the pre-trained visual attention model to distinguish the target pixels in the landscape image, and mark the pixel area formed by connecting the target pixels as a salient attention area;

[0061] Calculating the overlap between the salient attention area and the foreground area, and when the overlap is less than or equal to a preset overlap threshold, using a preconfigured genetic optimization algorithm to obtain an optimal blur radius of the background area;

[0062] The background area is blurred by Gaussian blurring according to the optimal blur radius to obtain a blurred landscape image.

[0063] Compared with the prior art, the present invention has the following beneficial effects:

[0064] The present application discloses an adaptive image blur processing system and method based on a deep convolutional neural network, comprising: using a pre-trained semantic segmentation model to segment the foreground area and the background area in a landscape image; inputting the landscape image into a pre-trained visual attention model to distinguish the target pixel points in the landscape image, and marking the pixel point area formed by connecting the target pixel points as a significant attention area; calculating the overlap between the significant attention area and the foreground area, and when the overlap is less than or equal to a preset overlap threshold, using a pre-configured genetic optimization algorithm to obtain the optimal blur radius of the background area; blurring the background area with Gaussian blur according to the optimal blur radius to obtain a blurred landscape image; based on the above process, the present invention is conducive to avoiding excessive or inappropriate blurring of the landscape image; in addition, by automatically screening the landscape image for blurring through the overlap, the present invention is conducive to improving the blurring efficiency and effectively responding to the real-time processing requirements of a large number of pictures on the tourism platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 A schematic diagram of the module structure of an adaptive image blur processing system based on a deep convolutional neural network provided by the present invention;

[0066] Figure 2 A flowchart of an adaptive image blur processing method based on a deep convolutional neural network provided by the present invention. DETAILED DESCRIPTION

[0067] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0068] Example 1

[0069] See also Figure 1 As shown, this embodiment discloses an adaptive image blur processing system based on a deep convolutional neural network, and the system includes:

[0070] The first segmentation module 101 is used to obtain the landscape image uploaded to the platform and segment the foreground area and the background area in the landscape image using a pre-trained semantic segmentation model;

[0071] It should be understood that: the platform may be a travel website or travel APP and other applications; there is a display page on such a platform for displaying uploaded scenic images of specific scenic spots; the sources of scenic images are diverse, including manual upload by platform administrators, acquisition from scenic spot websites, and upload by tourists;

[0072] In implementation, segmenting the foreground area and the background area in the landscape image includes:

[0073] The edge detection algorithm is used to identify multiple object contours in the landscape image, and the pixel area surrounded by each object contour is used as the verification area;

[0074] Wherein, the edge detection algorithm includes but is not limited to one of the Sobel algorithm, the Canny edge detection algorithm or the Prewitt algorithm;

[0075] Input each area to be verified into the pre-trained semantic segmentation model to obtain the category to which each area to be verified belongs;

[0076] Wherein, the categories include foreground categories and background categories;

[0077] The training logic of the semantic segmentation model is as follows:

[0078] Acquire historical semantic segmentation training data, and divide the historical semantic segmentation training data into a semantic segmentation training set and a semantic segmentation test set; the historical semantic segmentation training data includes a plurality of regions to be verified in the form of images and their corresponding category labels;

[0079] Constructing a first classifier, taking the region to be verified in the form of an image in the semantic segmentation training set as the input of the first classifier, and taking the category label in the semantic segmentation training set as the output of the first classifier, training the first classifier to obtain an initial semantic segmentation network;

[0080] Wherein, the first classifier is specifically a deep convolutional neural network (RCNN);

[0081] It should be noted that: the image-based verification area in the historical semantic segmentation training data is formed by the technicians performing the above-mentioned processing on the collected landscape images; the category labels in the historical semantic segmentation training data are manually annotated by the technicians or AI data annotators;

[0082] Using the semantic segmentation test set to verify the model of the initial semantic segmentation network, outputting an initial semantic segmentation network that is greater than or equal to a preset first test accuracy threshold as a trained semantic segmentation model;

[0083] Divide the corresponding area to be verified into a foreground area or a background area according to the category to which it belongs;

[0084] Exemplarily, for a landscape image, objects such as people, cars or the sky that appear in the image are regarded as the background, and areas other than the background are regarded as the foreground area, including buildings, forests and lakes. Therefore, assuming that there is a landscape image and it is identified that there are two areas to be verified, one is the sky and the other is the building, the area to be verified that is the sky is divided into the background area, and the area to be verified that is the building is divided into the foreground area.

[0085] The second segmentation module 102 is used to input the landscape image into a pre-trained visual attention model to distinguish target pixels in the landscape image, and mark the pixel area formed by connecting the target pixels as a salient attention area;

[0086] In implementation, the training logic of the visual attention model is as follows:

[0087] Acquire historical visual attention training data, and divide the historical visual attention training data into a visual attention training set and a visual attention test set; the historical visual attention training data includes a plurality of landscape images and a classification label for each pixel therein;

[0088] The classification labels include "0" and "1", "0" represents that the corresponding pixel is a non-target pixel, and "1" represents that the corresponding pixel is a target pixel;

[0089] It should be noted that: the above refers to the pixel points that are the focus of attention in the landscape image. In other words, each pixel point has an attention-attracting intensity. The higher the attention-attracting intensity, the easier it is for the corresponding pixel point to attract visual attention. When the attention-attracting intensity of a certain pixel point is greater than the intensity threshold of the threshold, the corresponding pixel point is marked as a target pixel point. Otherwise, the corresponding pixel point is marked as a non-target pixel point.

[0090] The generation logic of the classification label is as follows:

[0091] Get the attention intensity of each pixel in the landscape image;

[0092] Specifically, the process of obtaining the intensity of attracting attention is as follows:

[0093] Get the RGB value and brightness value of each pixel in the landscape image;

[0094] Substitute the RGB value and brightness value into the preset intensity calculation model to obtain the attention-attracting intensity of each pixel;

[0095] The expression of the strength calculation model is as follows:

[0096] ;

[0097] Where: Represents pixel The intensity of attracting attention, Represents pixel Color contrast, Represents pixel The brightness contrast ratio, and is a weight coefficient greater than zero, ;in, The calculation formula is: , The calculation formula is: ; Where: , G and B represent pixel points respectively Color values ​​in the red, green, and blue channels, , and Represented as pixels The color mean of the red, green, and blue channels in the neighborhood, , and Represented as pixels Luminance values ​​in the red, green, and blue channels, Represented as pixels The mean brightness of the red, green, and blue channels in the neighborhood;

[0098] It should be noted that the brightness mean is calculated based on the brightness mean of the red, green and blue channels in the neighborhood. For example, assuming there is a landscape image , and the size of the image is , using a matrix to represent the brightness distribution of the image P, we have: ;like The pixel point at the brightness value is the pixel point whose attention intensity is to be calculated, so the neighborhood of the pixel point is divided into 50, 60, 80, 70, 100, 40, 30 and By calculating the brightness value of each pixel in the neighborhood and The difference between the two values ​​is calculated, and all the differences are accumulated and averaged to obtain the brightness mean. Similarly, the logic for obtaining the color mean is the same, which will not be repeated here. For details, please refer to the explanation and logic of the brightness mean.

[0099] It can be understood that color contrast reflects the color difference between a pixel and its neighboring pixels; a larger color difference usually means that the pixel receives more attention and is more attractive; similarly, the same is true for high brightness contrast. People's eyes tend to be attracted by colorful or bright things. If the area attracted is not or is not only the subject, it is easy to cause the loss of visual beauty of the image;

[0100] Compare the attention-attracting intensity of each pixel with a preset attention-attracting intensity threshold;

[0101] If the attention-attracting intensity is greater than the attention-attracting intensity threshold, the corresponding pixel point is marked as a target pixel point;

[0102] If the attention-attracting intensity is less than or equal to the attention-attracting intensity threshold, the corresponding pixel point is marked as a non-target pixel point;

[0103] Constructing a second classifier, taking the landscape image in the visual attention training set as the input of the second classifier, and taking the classification label of each pixel in the visual attention training set as the output of the second classifier, training the second classifier to obtain an initial visual attention network;

[0104] Wherein, like the first classifier, the second classifier is specifically a deep convolutional neural network (RCNN);

[0105] The initial visual attention network is model verified using the visual attention test set, and an initial visual attention network that is greater than or equal to the second test accuracy threshold is output as the trained visual attention model.

[0106] The overlap judgment module 103 is used to calculate the overlap between the salient attention area and the foreground area, and when the overlap is less than or equal to a preset overlap threshold, obtain the optimal blur radius of the background area using a preconfigured genetic optimization algorithm;

[0107] In implementation, the calculation of the overlap between the salient attention area and the foreground area includes:

[0108] Acquire a pixel point set of a salient attention area, and mark the pixel point set of the salient attention area as a first pixel point set;

[0109] Acquire a pixel point set of the foreground area, and mark the pixel point set of the foreground area as a second pixel point set;

[0110] Calculate the intersection of the first pixel point set and the second pixel point set, and calculate the union of the first pixel point set and the second pixel point set;

[0111] The ratio of the intersection to the union is marked as the overlap between the salient attention area and the foreground area;

[0112] For example, assuming that the first pixel point set includes (1,1), (1,2), (2,2), (3,3) pixels, that is, A={(1,1), (1,2), (2,2), (3,3)}, and assuming that the second pixel point set includes (1,2), (2,1), (2,2), (3,4) pixels, then A∩B={(1,2), (2,2)}, the number of pixel points in the intersection |A∩B|=2, and A∪B={(1,1), (1,2), (2,2), (3,3), (2,1), (3,4)}, the number of pixel points in the union |A∪B|=6, therefore, ; If the preset overlap threshold is 0.5, the overlap between the salient attention area and the foreground area is less than the preset overlap threshold, which indicates that some elements in the background of the landscape image attract the user's attention to the foreground, and the background should be blurred to improve the image display effect;

[0113] In a specific implementation, when the overlap is greater than a preset overlap threshold, it is determined that the landscape image meets the set output standard, and the landscape image is directly displayed on the platform;

[0114] It can be understood that when the overlap between the salient attention area and the foreground area is less than or equal to the preset overlap threshold, it indicates that some elements in the background attract the user's attention to the foreground, which makes the foreground subject not well highlighted and reduces the layering of the image. Appropriate background blurring should be performed to effectively weaken the visual impact of these background elements to enhance the prominence of the foreground subject, and the platform display should be performed after blurring. On the contrary, when the overlap between the salient attention area and the foreground area is greater than the preset overlap threshold, it indicates that the original landscape image has a perfect composition, a prominent subject, and clear layers. No blurring is required and it can be directly uploaded to the platform for display.

[0115] In implementation, the method of obtaining the optimal blur radius of the background area by using a preconfigured genetic optimization algorithm includes:

[0116] a1: Initialize the population: randomly generate an original population, which contains K chromosomes, each chromosome represents a fuzzy radius, and K is an integer greater than zero;

[0117] a2: Fitness evaluation: Under each chromosome (i.e., each blur radius in the original population), obtain the average gradient change rate of the landscape image; input the average gradient change rate into the pre-constructed fitness function to calculate the fitness of each chromosome;

[0118] It should be understood that: when in a certain chromosome (i.e. blur radius), the system will automatically perform Gaussian blurring on the background area of ​​the landscape image according to the current chromosome (i.e. blur radius) to obtain the blurred landscape image in the experimental scene;

[0119] The logic for obtaining the average gradient change rate is as follows:

[0120] Extract the edge of the background area in the landscape image and calculate the gradient of each pixel in the edge in the horizontal and vertical directions;

[0121] The edge of the background area can be obtained by contour detection, threshold segmentation or edge detection; the gradients in the horizontal and vertical directions are calculated using the Sobel operator, respectively, where the gradient formula in the horizontal direction is: , the vertical gradient formula is: ; Where: and Respectively represent the horizontal and vertical gradients at the pixel point (x, y), is the partial derivative in the horizontal direction, which is used to describe the gradient of the pixel point (x, y) in the horizontal direction. It is the partial derivative in the vertical direction, which is used to describe the gradient of the pixel point (x, y) in the vertical direction;

[0122] Determine the gradient strength of each pixel in the edge portion according to the gradients in the horizontal and vertical directions;

[0123] The calculation formula of gradient strength is: ; Where: Represents the gradient intensity at the pixel point (x, y);

[0124] Obtain the adjacent pixels of each pixel in the edge part, as well as the gradient strength of the adjacent pixels;

[0125] The average gradient change rate of the edge part is calculated based on the gradient strength of each pixel and its adjacent pixels;

[0126] The calculation formula of the average gradient change rate is as follows:

[0127] ;

[0128] Where: Represents the gradient change rate at the pixel point (x, y), Represents the gradient strength of the pixel point (x, y) in the edge part, Represents the gradient strength at the adjacent pixel point (x+1,y), Represents the gradient strength at the adjacent pixel point (x, y+1); is the average gradient change rate, N is the number of pixels in the edge part, is the pixel set of the edge part;

[0129] It is clear that: if If the value is large, it means that the edge brightness or color of the foreground and background areas in the blurred landscape image changes rapidly, indicating that the edge is very sharp and the transition is not smooth. If it is very small, it means that the edge brightness or color of the foreground and background areas in the blurred landscape image changes very slowly, indicating that the edge transition is smooth;

[0130] The calculation formula of the pre-constructed fitness function is: ; Where: For fitness, is the average gradient change rate, e is a natural constant;

[0131] a3: Selection: Use the roulette wheel method to select two chromosomes with high fitness in the original population as the father and mother;

[0132] a4: Crossover: Perform crossover operation on the father and mother to generate new chromosomes;

[0133] It should be noted that the crossover operation on the paternal parent and the maternal parent is implemented based on a crossover operation, and the crossover operation includes but is not limited to one of a single-point crossover, a uniform crossover or a sequential crossover, etc.;

[0134] a5: Mutation: Perform mutation operation on the new chromosomes to obtain Y new chromosomes, combine the Y new chromosomes into a new population, replace the original population with the new population, and return to step a2;

[0135] In the genetic algorithm, the mutation operation is to introduce gene diversity and prevent the algorithm from falling into the local optimum; the mutation operation on the new chromosome is implemented by uniform mutation or Gaussian mutation;

[0136] a6: Repeat steps a2 to a5 above until the fitness of the chromosome in the original population or the new population is greater than or equal to the preset fitness threshold, or the number of iterations is greater than or equal to the preset maximum number of iterations threshold, and output the fuzzy radius represented by the corresponding chromosome as the optimal fuzzy radius;

[0137] For example: Assume that the maximum number of iterations is 100 times, and record the chromosome with the highest fitness in the current population and its fitness value after each iteration; if it is found that the fitness value has not changed significantly in a certain generation, it is considered that the convergence condition is met, the iteration is stopped, and the fuzzy radius represented by the corresponding chromosome is output as the optimal fuzzy radius.

[0138] A blur processing module 104 is used to perform a blur processing of the background area by Gaussian blur according to an optimal blur radius to obtain a blurred landscape image;

[0139] It can be understood that: by calculating and comparing the overlap between the significant attention area and the foreground area, the present invention is conducive to automatically determining whether the uploaded image has visual beauty and layering, and thus is conducive to targeted subsequent image blurring processing. Furthermore, by utilizing a genetic optimization algorithm to obtain the optimal blur radius and blurring the landscape image with the optimal blur radius, the present invention is conducive to ensuring that the image blurring processing is appropriate, helping to avoid the image losing its due natural feel, ensuring the visual effect of the uploaded image, and thus improving the overall experience of the tourism platform.

[0140] Example 2

[0141] See also Figure 2 As shown, based on the same inventive concept, this embodiment discloses an adaptive image blur processing method based on a deep convolutional neural network. For details not provided in this embodiment, please refer to the description of the relevant parts in Embodiment 1. The method includes:

[0142] S201: Obtain a landscape image uploaded to the platform, and use a pre-trained semantic segmentation model to segment a foreground area and a background area in the landscape image;

[0143] S202: inputting the landscape image into a pre-trained visual attention model to distinguish target pixels in the landscape image, and marking a pixel region formed by connecting the target pixels as a salient attention region;

[0144] S203: calculating the overlap between the salient attention area and the foreground area, and when the overlap is less than or equal to a preset overlap threshold, using a preconfigured genetic optimization algorithm to obtain an optimal blur radius for the background area;

[0145] S204: Performing Gaussian blurring on the background area according to the optimal blur radius to obtain a blurred landscape image.

[0146] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters, weights and thresholds in the formula are set by technicians in this field according to actual conditions.

[0147] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented by software, the above embodiments may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired network or a wireless network. The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media sets. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD) or a semiconductor medium. The semiconductor medium may be a solid-state hard disk.

Claims

1. An adaptive image blur processing system based on deep convolutional neural network, characterized in that: The system comprises: The first segmentation module is used to obtain the landscape image uploaded to the platform and segment the foreground area and background area of ​​the landscape image using a pre-trained semantic segmentation model; The second segmentation module is used to input the landscape image into the pre-trained visual attention model to distinguish the target pixels in the landscape image and mark the pixel area formed by connecting the target pixels as a salient attention area; Among them, the training logic of the visual attention model is as follows: Acquire historical visual attention training data, and divide the historical visual attention training data into a visual attention training set and a visual attention test set; the historical visual attention training data includes a plurality of landscape images and a classification label for each pixel therein; Wherein, the classification label includes "0" and "1", "0" represents that the corresponding pixel point is a non-target pixel point, and "1" represents that the corresponding pixel point is a target pixel point; The generation logic of the classification label is as follows: Get the attention intensity of each pixel in the landscape image; The process of obtaining the intensity of attracting attention is as follows: Get the RGB value and brightness value of each pixel in the landscape image; Substitute the RGB value and brightness value into the preset intensity calculation model to obtain the attention-attracting intensity of each pixel; The expression of the strength calculation model is as follows: ; Where: Represents pixel The intensity of attracting attention, Represents pixel Color contrast, Represents pixel The brightness contrast ratio, and is a weight coefficient greater than zero, ;in, The calculation formula is: , The calculation formula is: ; Where: , G and B represent pixel points respectively Color values ​​in the red, green, and blue channels, , and Represented as pixels The color mean of the red, green, and blue channels in the neighborhood, , and Represented as pixels Luminance values ​​in the red, green, and blue channels, Represented as pixels The mean brightness of the red, green, and blue channels in the neighborhood; Compare the attention-attracting intensity of each pixel with a preset attention-attracting intensity threshold; If the attention-attracting intensity is greater than the attention-attracting intensity threshold, the corresponding pixel point is marked as a target pixel point; If the attention-attracting intensity is less than or equal to the attention-attracting intensity threshold, the corresponding pixel point is marked as a non-target pixel point; Constructing a second classifier, taking the landscape image in the visual attention training set as the input of the second classifier, and taking the classification label of each pixel in the visual attention training set as the output of the second classifier, training the second classifier to obtain an initial visual attention network; Using the visual attention test set to perform model verification on the initial visual attention network, and outputting an initial visual attention network greater than or equal to the second test accuracy threshold as a trained visual attention model; An overlap judgment module is used to calculate the overlap between the salient attention area and the foreground area, and when the overlap is less than or equal to a preset overlap threshold, obtain an optimal blur radius of the background area using a preconfigured genetic optimization algorithm; Wherein, the method of obtaining the optimal blur radius of the background area by using a preconfigured genetic optimization algorithm includes: a1: Initialize the population: randomly generate an original population, which contains K chromosomes, each chromosome represents a fuzzy radius, and K is an integer greater than zero; a2: Fitness evaluation: Under each chromosome, obtain the average gradient change rate of the landscape image; input the average gradient change rate into the pre-built fitness function to calculate the fitness of each chromosome; The logic for obtaining the average gradient change rate is as follows: Extract the edge of the background area in the landscape image and calculate the gradient of each pixel in the edge in the horizontal and vertical directions; Determine the gradient strength of each pixel in the edge portion according to the gradients in the horizontal and vertical directions; Obtain the adjacent pixels of each pixel in the edge part, as well as the gradient strength of the adjacent pixels; The average gradient change rate of the edge part is calculated based on the gradient strength of each pixel and its adjacent pixels; The calculation formula of the average gradient change rate is as follows: ; Where: Represents the gradient change rate at the pixel point (x, y), Represents the gradient strength of the pixel point (x, y) in the edge part, Represents the gradient strength at the adjacent pixel point (x+1,y), Represents the gradient strength at the adjacent pixel point (x, y+1); is the average gradient change rate, N is the number of pixels in the edge part, is the pixel set of the edge part; The calculation formula of the pre-constructed fitness function is: ; Where: For fitness, is the average gradient change rate, e is a natural constant; a3: Selection: Use the roulette wheel method to select two chromosomes with high fitness in the original population as the father and mother; a4: Crossover: Perform crossover operation on the father and mother to generate new chromosomes; a5: Mutation: Perform mutation operation on the new chromosomes to obtain Y new chromosomes, combine the Y new chromosomes into a new population, replace the original population with the new population, and return to step a2; a6: Repeat steps a2 to a5 above until the fitness of the chromosome in the original population or the new population is greater than or equal to the preset fitness threshold, or the number of iterations is greater than or equal to the preset maximum number of iterations threshold, and output the fuzzy radius represented by the corresponding chromosome as the optimal fuzzy radius; The blur processing module is used to perform Gaussian blur blur processing on the background area according to the optimal blur radius to obtain a blurred landscape image.

2. The adaptive image blur processing system based on deep convolutional neural network according to claim 1, characterized in that: The step of segmenting the foreground area and the background area in the landscape image includes: The edge detection algorithm is used to identify multiple object contours in the landscape image, and the pixel area surrounded by each object contour is used as the verification area; Input each area to be verified into the pre-trained semantic segmentation model to obtain the category to which each area to be verified belongs; Wherein, the categories include foreground categories and background categories; The training logic of the semantic segmentation model is as follows: Acquire historical semantic segmentation training data, and divide the historical semantic segmentation training data into a semantic segmentation training set and a semantic segmentation test set; the historical semantic segmentation training data includes a plurality of regions to be verified in the form of images and their corresponding category labels; Constructing a first classifier, taking the region to be verified in the form of an image in the semantic segmentation training set as the input of the first classifier, and taking the category label in the semantic segmentation training set as the output of the first classifier, training the first classifier to obtain an initial semantic segmentation network; Using the semantic segmentation test set to verify the model of the initial semantic segmentation network, outputting an initial semantic segmentation network that is greater than or equal to a preset first test accuracy threshold as a trained semantic segmentation model; The corresponding area to be verified is divided into a foreground area or a background area according to its category.

3. The adaptive image blur processing system based on deep convolutional neural network according to claim 2, characterized in that: The calculating the overlap between the salient attention area and the foreground area includes: Acquire a pixel point set of a salient attention area, and mark the pixel point set of the salient attention area as a first pixel point set; Acquire a pixel point set of the foreground area, and mark the pixel point set of the foreground area as a second pixel point set; Calculate the intersection of the first pixel point set and the second pixel point set, and calculate the union of the first pixel point set and the second pixel point set; The ratio of the intersection to the union is marked as the overlap between the salient attention region and the foreground region.

4. The adaptive image blur processing system based on deep convolutional neural network according to claim 3, characterized in that: When the degree of overlap is greater than a preset threshold value of the degree of overlap, it is determined that the landscape image meets the set output standard, and the landscape image is directly displayed on the platform.

5. An adaptive image blur processing method based on deep convolutional neural network, characterized in that: The method is implemented based on the adaptive image blur processing system based on a deep convolutional neural network according to any one of claims 1 to 4, and the method comprises: Obtain the landscape image uploaded to the platform, and use the pre-trained semantic segmentation model to segment the foreground and background areas in the landscape image; Input the landscape image into the pre-trained visual attention model to distinguish the target pixels in the landscape image, and mark the pixel area formed by connecting the target pixels as a salient attention area; Calculating the overlap between the salient attention area and the foreground area, and when the overlap is less than or equal to a preset overlap threshold, using a preconfigured genetic optimization algorithm to obtain an optimal blur radius for the background area; The background area is blurred by Gaussian blurring according to the optimal blur radius to obtain a blurred landscape image.

Images