Flower exhibition image stylization system based on multi-sampling anti-aliasing

Abstract

The invention discloses a multi-sampling anti-aliasing-based flower exhibition image stylization system, and relates to the technical field of image processing, the system obtains multi-angle and multi-period image data through an image acquisition module and automatically divides an image acquisition area, and an image preprocessing module performs multi-sampling anti-aliasing processing to improve the image quality. The style feature extraction module classifies target styles and forms a style set, the image content analysis module predicts viewing satisfaction, the style fusion module determines a stylization scheme, and the system further comprises an effect evaluation module and a visual display module which are respectively used for evaluating the effect of the stylization scheme and displaying the scheme and an evaluation result. According to the system, the digital and artistic reproduction of the flower exhibition image is realized, and the viewing experience and the accuracy of satisfaction prediction are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and more specifically, to a flower exhibition image stylization system based on multi-sampling anti-aliasing. Background Art

[0002] With the rapid development of computer technology, image processing technology has been widely used in various fields. In the field of flower exhibition image stylization, traditional image processing technologies have many deficiencies. First, the limitations of image acquisition devices result in differences in the acquired image data at different angles and time periods, affecting the integrity and accuracy of the images. Second, the image preprocessing technology is not perfect enough to effectively improve the image quality. Especially in anti-aliasing processing, traditional anti-aliasing technologies such as supersampling anti-aliasing and fast approximate anti-aliasing can reduce the aliasing phenomenon to a certain extent, but they consume a large amount of computing resources and the effect is not ideal enough. In addition, the existing technologies also have deficiencies in style feature extraction and fusion, unable to accurately extract and classify the features of multiple target styles, resulting in a single stylization effect and unable to meet the users' needs for diverse styles. Finally, the existing technologies lack in-depth analysis and evaluation of the image content and cannot dynamically adjust the stylization scheme according to the viewers' feedback, affecting the viewing experience.

[0003] Therefore, the existing flower exhibition image stylization technologies cannot accurately extract multiple style features, have a single stylization effect, and lack the ability of dynamic adjustment. Summary of the Invention

[0004] In order to overcome the problems that the existing flower exhibition image stylization technologies cannot accurately extract multiple style features, have a single stylization effect, and lack the ability of dynamic adjustment, etc., the present invention designs a flower exhibition image stylization system based on multi-sampling anti-aliasing, which can effectively solve the above technical problems.

[0005] To solve the above technical problems, the technical solution of the present invention is as follows:

[0006] A flower exhibition image stylization system based on multi-sampling anti-aliasing, comprising:

[0007] An image acquisition module, configured to acquire a plurality of image data of a flower exhibition site at different angles and different time periods, and automatically divide the flower exhibition site into a plurality of image acquisition areas according to the shooting parameters of the image data, and each of the plurality of image acquisition areas includes a plurality of images;

[0008] An image preprocessing module, configured to acquire the image attributes of the plurality of images included in each of the plurality of image acquisition areas, and perform multi-sampling anti-aliasing processing on the plurality of images according to the image attributes to improve the image quality;

[0009] A style feature extraction module, which is used to obtain the style information of multiple target style images, classify the multiple target styles according to the style information, and obtain multiple style sets, where the number of the style sets is the same as the number of the image acquisition areas;

[0010] An image content analysis module, which is used to obtain the viewing information corresponding to multiple images respectively included in the multiple image acquisition areas, and predict the viewing satisfaction of the flowers that need to be newly displayed or the layout of the flowers that need to be adjusted in different exhibition areas of the flower show according to the viewing information;

[0011] A style fusion module, which is used to respectively obtain multiple stylized video information of the multiple image acquisition areas, analyze the multiple stylized video information to obtain the daily regional attention, regional viewing duration and target preferences, where the target preferences are used to represent the attitudes of each viewer towards the multiple target styles, and the attitudes include like, dislike and neutral; it is also used to determine the stylization scheme of the multiple images according to the viewing satisfaction, the daily regional attention, the regional viewing duration and the target preferences, and the stylization scheme includes the corresponding relationship between the multiple image acquisition areas and the multiple style sets, and the specific stylization effects of the multiple images;

[0012] An effect evaluation module, which is used to evaluate the actual effect of the stylization scheme and adjust the parameters and strategies of style fusion according to the evaluation results;

[0013] A visualization display module, which is used to display the stylization scheme and the evaluation results.

[0014] Preferably, the image acquisition module includes:

[0015] An image data acquisition unit, which specifically acquires image data of different angles and different time periods at the flower show site by combining an unmanned aerial vehicle and a fixed camera;

[0016] A region division unit, which specifically performs cluster analysis on the shooting parameters of the image data, classifies the image data, and determines the optimal number of clusters through the silhouette coefficient to obtain the multiple image acquisition areas, where the number of the multiple image acquisition areas is the same as the optimal number of clusters.

[0017] Preferably, the image preprocessing module includes:

[0018] An image attribute acquisition unit, which is used to acquire the image attributes of the multiple images, and the image attributes include resolution, color mode and edge complexity;

[0019] An anti-aliasing processing unit, which dynamically adjusts the distribution and quantity of sampling points according to the edge complexity and resolution of the image, and performs anti-aliasing processing on the image using an adaptive multi-sampling algorithm.

[0020] Preferably, the style feature extraction module includes:

[0021] A style information acquisition unit, which is used to respectively acquire the style information of the multiple target style images through big data technology, and the style information is text and image feature information;

[0022] A style feature extraction unit, which is used to perform feature extraction on the target style image according to the style information through a feature extraction algorithm, and obtain style feature vectors corresponding to the multiple target styles respectively, and the style feature vectors are used to represent the features of each target style;

[0023] A style set determination unit, which performs clustering analysis on the style feature vectors corresponding to the multiple target styles respectively, classifies the multiple target styles according to the optimal clustering quantity, and obtains the multiple style sets, and the number of the multiple style sets is the same as the optimal clustering quantity.

[0024] Preferably, the image content analysis module includes:

[0025] An appreciation information acquisition unit, which is used to obtain the appreciation information corresponding to the multiple images respectively through big data technology, and the appreciation information includes the number of viewers, the staying time, and the evaluation feedback;

[0026] A satisfaction calculation unit, which is used to calculate the appreciation satisfaction corresponding to the multiple images respectively, and each of the multiple images corresponds to an appreciation satisfaction;

[0027] A satisfaction prediction unit, which is used to train a deep learning network model according to the style feature vectors corresponding to the multiple images respectively, the position information of the images in the exhibition area, and the appreciation satisfaction, and obtain a satisfaction prediction model that can output the appreciation satisfaction of the newly added display flowers or the flowers that need to be adjusted in layout in different exhibition areas of the flower exhibition; taking the new style feature vectors and position information corresponding to the newly added flowers or the flowers that need to be adjusted in layout as the input of the satisfaction prediction model, and the satisfaction prediction model outputs the appreciation satisfaction of the newly added flowers or the flowers that need to be adjusted in layout.

[0028] Preferably, the style fusion module includes:

[0029] A stylized video information acquisition unit is configured to respectively acquire multiple pieces of stylized video information of the multiple image acquisition areas; and is further configured to respectively perform frame splitting processing on the multiple pieces of stylized video information to obtain multiple split-frame images, where each of the multiple split-frame images corresponds to a time point, and the multiple pieces of stylized video information of the multiple image acquisition areas respectively include the multiple split-frame images;

[0030] An area attention degree determination unit, based on image recognition technology, respectively performs face recognition and number statistics on the multiple split-frame images to obtain the number of viewers appearing in the multiple split-frame images, and obtains the number of viewers corresponding to each of the multiple split-frame images according to the number of viewers; according to the number of viewers corresponding to each of the multiple split-frame images, calculates the daily number of viewers corresponding to each of the multiple pieces of stylized video information of the multiple image acquisition areas; performs an average value calculation on the daily number of viewers corresponding to each of the multiple pieces of stylized video information to obtain the daily average number of viewers corresponding to each of the multiple image acquisition areas, and determines the daily average number of viewers as the daily area attention degree;

[0031] An area viewing duration determination unit, specifically based on image recognition technology, respectively performs face recognition on the multiple split-frame images, and determines the maximum duration of each viewer's continuous appearance at multiple time points in the multiple split-frame images of the multiple image acquisition areas as the viewing duration of each viewer in the multiple image acquisition areas; determines the time point corresponding to the first appearance of each viewer in the multiple split-frame images of the multiple image acquisition areas as the first time point, determines the time point corresponding to the last appearance of each viewer in the multiple split-frame images of the multiple image acquisition areas as the second time point, calculates the time difference between the first time point and the second time point, and determines the time difference as the maximum duration;

[0032] A target preference determination unit, through an object detection algorithm, respectively analyzes and processes the multiple split-frame images to respectively obtain the style positions of the target styles appearing in the multiple split-frame images; then, through a gaze direction recognition model, recognizes the gaze directions of the viewers appearing in the multiple split-frame images; and according to the style positions and the gaze directions, determines the target styles watched by the viewers appearing in the multiple split-frame images; finally, through a preference recognition model, recognizes the target preference feature vectors of the viewers appearing in the multiple split-frame images, determines the preference corresponding to the target preference feature vectors as the target preference, and associates the target preference with the target style;

[0033] A stylization scheme determination unit for determining a stylization scheme for the multiple images according to the viewing satisfaction, the daily area attention, the area viewing duration, and the target preference.

[0034] Preferably, for the gaze direction recognition model, specifically, based on a plurality of first training images with actual gaze direction labels, the deep learning network is trained to obtain the gaze direction recognition model that can output the gaze direction of the viewer appearing in the multiple framed images; for the preference recognition model, specifically, based on a plurality of second training images with preference feature vectors, the deep learning network is trained to obtain the preference recognition model that can output the preference of the viewer appearing in the multiple framed images; each preference feature vector corresponds to a preference label.

[0035] Preferably, the stylization scheme determination unit includes:

[0036] A comprehensive preference determination subunit, specifically performing mean pooling on the multiple target preference feature vectors respectively corresponding to the multiple target preferences to obtain a mean vector, calculating the similarity between the mean vector and each of the multiple preference feature vectors, and determining the preference label corresponding to the preference feature vector with the highest similarity to the mean vector as the comprehensive preference;

[0037] A correspondence determination subunit, normalizing the daily area attention and the area viewing duration for the regional feature vectors respectively constituting the multiple image acquisition areas; then performing weight allocation on the daily area attention and the area viewing duration after normalization through a graph attention network; finally, performing weight calculation according to the weights respectively corresponding to the daily area attention and the area viewing duration to obtain the regional feature vectors respectively corresponding to the multiple image acquisition areas; normalizing the viewing satisfaction and the mean vector for obtaining the comprehensive preference for the image feature vectors respectively constituting the multiple images; then performing weight allocation on the viewing satisfaction and the mean vector after normalization through a graph attention network; finally, performing weight calculation according to the weights respectively corresponding to the viewing satisfaction and the mean vector to obtain the image feature vectors respectively corresponding to the multiple images; determining the correspondence, calculating the similarity between each regional feature vector and each of the multiple set feature vectors to obtain the similarity between each regional feature vector and each set feature vector, corresponding the regional feature vector with the highest similarity and the set feature vector, and corresponding the image acquisition area corresponding to the regional feature vector and the style set corresponding to the set feature vector;

[0038] The stylization effect determination subunit hierarchically divides the multiple image acquisition regions into multiple stylization levels, and the multiple stylization levels are sorted in descending order of salience; then calculates the norms of the image feature vectors corresponding to the multiple images respectively, classifies them according to the norms of the image feature vectors to obtain multiple image stylization combinations, and the number of the image stylization combinations is the same as the number of the stylization levels; and calculates the total norm of the multiple image stylization combinations, and the multiple image stylization combinations are sorted in descending order according to the total norm. Finally, determines the correspondence between the multiple image stylization combinations and the multiple stylization levels. The corresponding rule is to correspond the image stylization combination with the largest total norm to the highest stylization level, and successively correspond the remaining image stylization combinations to the remaining stylization levels according to the sorting of the image stylization combinations and the sorting of the stylization levels.

[0039] Preferably, the effect evaluation module includes:

[0040] An evaluation index setting unit for setting an index for evaluating the effect of the stylization scheme;

[0041] An effect evaluation unit for evaluating the actual effect of the stylization scheme according to the evaluation index;

[0042] A parameter adjustment unit for adjusting the parameters and strategies of style fusion according to the evaluation result and feeding them back to the style fusion module for optimization.

[0043] Preferably, the visualization display module includes:

[0044] A scheme display unit for displaying the stylization scheme, including the correspondence between the image acquisition region and the style set, and the specific stylization effect of the image;

[0045] An evaluation result display unit for displaying the evaluation result of the stylization scheme.

[0046] Compared with the prior art, the beneficial effects of the present invention are as follows: The present invention obtains image data from different angles and time periods through the image acquisition module, and divides the flower exhibition site into multiple image acquisition areas according to the shooting parameters, ensuring the comprehensiveness and accuracy of the image data. The image preprocessing module performs multiple sampling anti-aliasing processing on the acquired images, improving the image quality and solving the problem of serious image aliasing. The style feature extraction module accurately extracts the features of multiple target styles through big data technology and feature extraction algorithms, and classifies them to obtain multiple style sets, solving the problem of being unable to accurately extract the features of multiple styles. The image content analysis module obtains viewing information through big data technology and predicts the viewing satisfaction, providing an important basis for determining the stylization scheme. The style fusion module comprehensively considers the viewing satisfaction, regional attention, viewing duration, and target preferences to determine the stylization scheme of the image, making the stylization effect more diverse and personalized, and solving the problem of single stylization effect. The effect evaluation module adjusts the parameters and strategies of style fusion according to the evaluation results, enabling the system to have dynamic adjustment capabilities and enhancing the viewing experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only exemplary, and for those of ordinary skill in the art, without creative efforts, other implementation drawings can be obtained based on the provided drawings.

[0048] Figure 1 It is the overall system structure block diagram of the present invention;

[0049] Figure 2 It is the structure block diagram of the stylization scheme determination unit of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0050] The drawings are only for exemplary illustration and cannot be construed as a limitation of this patent;

[0051] In order to better illustrate this embodiment, some components in the drawings will be omitted, enlarged, or reduced, which does not represent the size of the actual product;

[0052] For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0053] The following will further explain the technical solutions of the present invention in conjunction with the drawings and embodiments.

[0054] Embodiment

[0055] A flower exhibition image stylization system based on multiple sampling anti-aliasing, please refer to Figure 1-2 , including:

[0056] An image acquisition module, configured to obtain a plurality of image data of the flower exhibition site at different angles and different time periods, and automatically divide the flower exhibition site into a plurality of image acquisition areas according to the shooting parameters of the image data, wherein the plurality of image acquisition areas respectively include a plurality of images;

[0057] An image preprocessing module, configured to obtain the image attributes of the plurality of images respectively included in the plurality of image acquisition areas, and perform multi-sampling anti-aliasing processing on the plurality of images according to the image attributes to improve the image quality;

[0058] A style feature extraction module, configured to obtain the style information of a plurality of target style images, classify the plurality of target styles according to the style information to obtain a plurality of style sets, and the number of the style sets is the same as the number of the image acquisition areas;

[0059] An image content analysis module, configured to obtain the viewing information corresponding to the plurality of images respectively included in the plurality of image acquisition areas, and predict the viewing satisfaction of the flowers that need to be newly displayed or the layout of the flowers that need to be adjusted in different exhibition areas of the flower exhibition according to the viewing information;

[0060] A style fusion module, configured to respectively obtain a plurality of stylized video information of the plurality of image acquisition areas, analyze the plurality of stylized video information to obtain the daily regional attention, regional viewing duration and target preference, wherein the target preference is used to represent the attitude of each viewer towards the plurality of target styles, and the attitude includes like, dislike and neutral; and is further configured to determine a stylization scheme for the plurality of images according to the viewing satisfaction, the daily regional attention, the regional viewing duration and the target preference, wherein the stylization scheme includes the corresponding relationship between the plurality of image acquisition areas and the plurality of style sets, and the specific stylization effects of the plurality of images;

[0061] An effect evaluation module, configured to evaluate the actual effect of the stylization scheme, and adjust the parameters and strategies of style fusion according to the evaluation result;

[0062] A visualization display module, configured to display the stylization scheme and the evaluation result.

[0063] The image acquisition module includes:

[0064] An image data acquisition unit, specifically configured to collect image data of the flower exhibition site at different angles and different time periods by combining an unmanned aerial vehicle and a fixed camera;

[0065] The area division unit first performs clustering analysis on the shooting parameters of the image data, classifies the image data, and determines the optimal number of clusters through the silhouette coefficient for the clustering analysis, obtaining the multiple image acquisition areas, where the number of the multiple image acquisition areas is the same as the optimal number of clusters.

[0066] The image data acquisition unit uses a combination of drones and fixed cameras to collect images of the flower show site. The drone takes high-altitude photos on a preset flight route to ensure full coverage of the flower show. According to the exhibition area and the performance of the drone, the flight path and shooting height of the drone are planned. For example, starting from the center point of the flower show, the flight route expands outward in a spiral shape, and the shooting height is maintained at a fixed height from the ground. At the same time, fixed cameras are pre-installed near multiple key exhibition areas and entrances and exits. A number of fixed cameras are configured, and each camera has a fixed shooting angle and focal length to achieve all-weather and multi-period shooting, recording the light changes and flower states of the flower show at different times, such as the quiet state before the park opens in the morning, the gradual increase in the number of tourists around a certain moment in the morning, the strong sunlight period at noon, and the sunset period around a certain moment in the afternoon.

[0067] The area division unit performs clustering analysis on the image data collected by the drone and the fixed cameras according to the shooting parameters. First, it extracts features such as the shooting time, shooting angle, shooting focal length, and light intensity of the image, and then uses the K-means clustering algorithm to classify the image data, sets the number of clusters, evaluates the clustering results through the silhouette coefficient, and finally determines the image acquisition areas with the same number as the number of clusters. The areas are the core flower display area, the tourist gathering service area, and the flower nursery cultivation area, each containing multiple images. For example, the core flower display area includes images of the rose exhibition area, the tulip exhibition area, etc.

[0068] The image preprocessing module includes:

[0069] The image attribute acquisition unit is used to acquire the image attributes of the multiple images, and the image attributes include resolution, color mode, and edge complexity.

[0070] The anti-aliasing processing unit dynamically adjusts the distribution and number of sampling points according to the edge complexity and resolution of the image, and performs anti-aliasing processing on the image using the adaptive multi-sampling algorithm.

[0071] For each collected image, the image attribute acquisition unit acquires its resolution, color mode, and edge complexity. Taking an image with a resolution of 1920×1080 as an example, the color mode is RGB, and the edge detection algorithm is used to calculate the proportion of the number of edge pixels to the total number of pixels, such as 12%, as the value of the edge complexity.

[0072] The anti-aliasing processing unit dynamically adjusts the distribution and quantity of sampling points according to the edge complexity and resolution of the image. For images with higher edge complexity and lower resolution, it increases the number of sampling points and adopts an adaptive multi-sampling algorithm. For example, it uses 4-sample anti-aliasing in the edge area and 2-sample anti-aliasing in the non-edge area to balance the anti-aliasing effect and computational cost and improve the overall quality of the image.

[0073] The style feature extraction module includes:

[0074] A style information acquisition unit, which is used to respectively acquire the style information of the multiple target style images through big data technology. The style information is text and image feature information;

[0075] A style feature extraction unit, which is used to extract features from the target style images according to the style information through a feature extraction algorithm to obtain style feature vectors corresponding to the multiple target styles respectively. The style feature vectors are used to represent the features of each target style;

[0076] A style set determination unit performs clustering analysis on the style feature vectors corresponding to the multiple target styles respectively, classifies the multiple target styles according to the optimal number of clusters, and obtains the multiple style sets. The number of the multiple style sets is the same as the optimal number of clusters.

[0077] The style information acquisition unit collects the style information of multiple target style images through big data technology. For example, it obtains 5000 flower images of "fresh and natural style" and 2000 flower images of "romantic and dreamy style" from social media platforms, photography websites, etc. as style information.

[0078] The style feature extraction unit uses a feature extraction algorithm, such as the VGG16 model in a convolutional neural network (CNN), to extract features from the target style images. It inputs each target style image into the VGG16 model, obtains its feature map in the last convolutional layer, and then unfolds these feature maps into vectors to obtain the style feature vectors corresponding to each target style.

[0079] The style set determination unit performs clustering analysis on the extracted style feature vectors. Similarly, it uses the K-means algorithm to determine the number of clusters according to the silhouette coefficient, which is the same as the number of image acquisition regions. It clusters 5000 flower images of "fresh and natural style" and 2000 flower images of "romantic and dreamy style" to form the same number of style sets as the number of clusters. Each style set contains different proportions of fresh and natural style and romantic and dreamy style images to match the corresponding image acquisition regions.

[0080] The image content analysis module includes:

[0081] An ornamental information acquisition unit for obtaining the ornamental information corresponding to each of the multiple images through big data technology, where the ornamental information includes the number of viewers, the staying time, and evaluation feedback;

[0082] A satisfaction calculation unit for calculating the ornamental satisfaction corresponding to each of the multiple images, where each of the multiple images corresponds to one ornamental satisfaction;

[0083] A satisfaction prediction unit for training a deep learning network model based on the style feature vectors, the position information of the images in the exhibition area, and the ornamental satisfaction corresponding to each of the multiple images, to obtain a satisfaction prediction model that can output the ornamental satisfaction of the newly added display flowers or the flowers that need to be adjusted in layout in different exhibition areas of the flower exhibition; using the new style feature vectors and position information corresponding to the newly added flowers or the flowers that need to be adjusted in layout as the input of the satisfaction prediction model, and the satisfaction prediction model outputs the ornamental satisfaction of the newly added flowers or the flowers that need to be adjusted in layout.

[0084] The ornamental information acquisition unit uses the image statistics and user feedback collection methods in big data technology to count the number of viewers, the staying time of each image, and the evaluation feedback of tourists on the flowers. For example, through the face detection technology of tourists in the images captured by fixed cameras, the number of tourists in each image area is counted and recorded in minutes. At the same time, a questionnaire survey and an online feedback platform are set up at the flower exhibition site to collect evaluation information such as the satisfaction and recommendation degree of tourists on the flowers.

[0085] For each image, the satisfaction calculation unit calculates the corresponding ornamental satisfaction value according to its number of viewers, staying time, and evaluation feedback, combined with the pre-set weights. For example, the proportion of the number of viewers is 40%, the proportion of the staying time is 30%, and the proportion of the evaluation feedback is 30%. The range of the ornamental satisfaction value is 0-100 points. For example, the number of viewers in a certain image area at a certain time period is 100 people, the staying time is an average of 5 minutes, and the evaluation feedback is that 80% of the tourists are satisfied. Then the ornamental satisfaction score of this image is 0.4×100 + 0.3×5 + 0.3×80 = 77 points.

[0086] The satisfaction prediction unit adopts a deep learning model, such as an LSTM network, inputs the style feature vectors of the flowers, the position information in the exhibition area, such as coordinate positions, relative distances from other flowers, etc., and the current ornamental satisfaction, and trains the network model. When it is necessary to predict the ornamental satisfaction of newly added flowers or flowers that need to be adjusted in layout, the style feature vectors and position information of the new flowers are input, and the model outputs its expected ornamental satisfaction to assist in optimizing the layout of the flower exhibition.

[0087] The style fusion module includes:

[0088] A stylized video information acquisition unit is configured to respectively acquire multiple pieces of stylized video information of the multiple image acquisition regions; and is further configured to perform frame splitting processing on the multiple pieces of stylized video information respectively to obtain multiple split-frame images, where each of the multiple split-frame images corresponds to a time point, and the multiple pieces of stylized video information of the multiple image acquisition regions respectively include the multiple split-frame images;

[0089] A regional attention degree determination unit, based on image recognition technology, respectively performs face recognition and number statistics on the multiple split-frame images to obtain the number of viewers appearing in the multiple split-frame images, and obtains the number of viewers corresponding to each of the multiple split-frame images according to the number of viewers; according to the number of viewers corresponding to each of the multiple split-frame images, calculates the daily number of viewers corresponding to each of the multiple pieces of stylized video information of the multiple image acquisition regions; performs an average value calculation on the daily number of viewers corresponding to each of the multiple pieces of stylized video information to obtain the daily average number of viewers corresponding to each of the multiple image acquisition regions, and determines the daily average number of viewers as the daily regional attention degree;

[0090] A regional viewing duration determination unit, specifically based on image recognition technology, respectively performs face recognition on the multiple split-frame images, and determines the maximum duration of each viewer appearing continuously at multiple time points in the multiple split-frame images of the multiple image acquisition regions as the viewing duration of each viewer in the multiple image acquisition regions; determines the time point corresponding to the first appearance of each viewer in the multiple split-frame images of the multiple image acquisition regions as the first time point, determines the time point corresponding to the last appearance of each viewer in the multiple split-frame images of the multiple image acquisition regions as the second time point, calculates the time difference between the first time point and the second time point, and determines the time difference as the maximum duration;

[0091] A target preference determination unit analyzes and processes the multiple split-frame images respectively through an object detection algorithm to respectively obtain the style positions of the target style appearing in the multiple split-frame images; then, through a gaze direction recognition model, recognizes the gaze directions of the viewers appearing in the multiple split-frame images; and according to the style positions and the gaze directions, determines the target style watched by the viewers appearing in the multiple split-frame images; finally, through a preference recognition model, recognizes the target preference feature vector of the viewers appearing in the multiple split-frame images, determines the preference corresponding to the target preference feature vector as the target preference, and associates the target preference with the target style;

[0092] A stylization scheme determination unit, configured to determine a stylization scheme for the multiple images according to the viewing satisfaction degree, the daily regional attention degree, the regional viewing duration, and the target preference.

[0093] The gaze direction recognition model specifically trains a deep learning network according to multiple first training images with actual gaze direction labels to obtain the gaze direction recognition model that can output the gaze direction of the viewer appearing in the multiple framed images; the preference recognition model specifically trains a deep learning network according to multiple second training images with preference feature vectors to obtain the preference recognition model that can output the preference of the viewer appearing in the multiple framed images; each preference feature vector corresponds to a preference label.

[0094] The stylization scheme determination unit includes:

[0095] A comprehensive preference determination subunit specifically performs mean pooling on the multiple target preference feature vectors respectively corresponding to the multiple target preferences to obtain a mean vector, calculates the similarity between the mean vector and each of the multiple preference feature vectors, and determines the preference label corresponding to the preference feature vector with the highest similarity to the mean vector as the comprehensive preference;

[0096] A correspondence determination subunit normalizes the daily regional attention degree and the regional viewing duration for the regional feature vectors respectively constituting the multiple image acquisition regions; then assigns weights to the daily regional attention degree and the regional viewing duration after normalization through a graph attention network; finally, performs weighted calculation according to the weights respectively corresponding to the daily regional attention degree and the regional viewing duration to obtain the regional feature vectors respectively corresponding to the multiple image acquisition regions; normalizes the viewing satisfaction degree and the mean vector for obtaining the comprehensive preference for the image feature vectors respectively constituting the multiple images; then assigns weights to the viewing satisfaction degree and the mean vector after normalization through a graph attention network; finally, performs weighted calculation according to the weights respectively corresponding to the viewing satisfaction degree and the mean vector to obtain the image feature vectors respectively corresponding to the multiple images; determines the correspondence, calculates the similarity between each regional feature vector and each set of feature vectors respectively, obtains the similarity between each regional feature vector and each set of feature vectors, corresponds the regional feature vector with the highest similarity and the set of feature vectors, and corresponds the image acquisition region corresponding to the regional feature vector and the style set corresponding to the set of feature vectors;

[0097] The stylization effect determination subunit hierarchically divides the multiple image acquisition regions into multiple stylization levels, and sorts the multiple stylization levels in descending order of prominence; then calculates the norms of the image feature vectors corresponding to the multiple images respectively, and classifies them according to the norms of the image feature vectors to obtain multiple image stylization combinations, where the number of the image stylization combinations is the same as the number of the stylization levels; and calculates the total norm of the multiple image stylization combinations, and sorts the multiple image stylization combinations in descending order according to the total norm. Finally, it determines the correspondence between the multiple image stylization combinations and the multiple stylization levels. The corresponding rule is to correspond the image stylization combination with the largest total norm to the highest stylization level, and sequentially correspond the remaining image stylization combinations to the remaining stylization levels according to the sorting of the image stylization combinations and the sorting of the stylization levels.

[0098] The stylized video information acquisition unit acquires stylized video information in each image acquisition region and performs frame-by-frame processing on each video. For example, a video with a fixed number of frames per second is decomposed into images corresponding to multiple time points. For a stylized video in the core area of flower display, it is decomposed into each frame image, and each frame image records the changes of flowers, tourists and the on-site environment at a certain time point.

[0099] The regional attention degree determination unit uses the face recognition algorithm in image recognition technology to detect and track faces in each frame of image, and counts the number of tourists in each frame of image. For example, if an average of 50 tourists are recognized in each frame of image, then the number of tourists at this time point is 50. Then, it summarizes according to the time periods of each day, calculates the number of daily viewers, and takes the average value to obtain the daily regional attention degree. Suppose the average number of tourists in a certain region per hour segment on a certain day is 200, then the daily regional attention degree is 200.

[0100] The regional viewing duration determination unit uses the face recognition algorithm to record the appearance time of each tourist in the video. For example, if tourist A first appears at 9:00 and the last appears at 9:30 in the image acquisition region A, then his viewing duration is 30 minutes. At the same time, if a tourist appears at multiple time points continuously in the region, the maximum duration is used as his viewing duration.

[0101] The target preference determination unit identifies the target style positions in each frame of the image through an object detection algorithm (such as YOLOv5), such as detecting the area coordinates of the romantic and dreamy style flowers in the frame image. It uses a pre-trained gaze direction recognition model (based on a convolutional neural network and trained with face images annotated with gaze directions) to identify the gaze direction of the tourist, determine the target style being viewed by the tourist. Finally, a preference recognition model, such as a classification model based on Softmax regression, uses the training data of tourist gaze images with preference labels to classify the viewing preferences of the tourist, identify preferences such as like, dislike or neutral of the tourist, and associate this preference with the corresponding target style.

[0102] The stylization scheme determination unit determines the stylization scheme by comprehensively considering the viewing satisfaction, daily area attention 200, area viewing duration and target preference information (the proportion of tourists who like the romantic and dreamy style and the proportion of neutral ones). For example, it corresponds the image acquisition area of the core flower display area to the style set of the romantic and dreamy style, adds effects such as a dreamy background and a soft filter to the flower images in this area to enhance the romantic atmosphere. At the same time, according to the tourist preferences, it adjusts the flower layout and increases the display of favorite romantic flower species.

[0103] The said effect evaluation module includes:

[0104] An evaluation index setting unit, which is used to set the indexes for evaluating the effect of the stylization scheme;

[0105] An effect evaluation unit, which is used to evaluate the actual effect of the stylization scheme according to the said evaluation indexes;

[0106] A parameter adjustment unit, which is used to adjust the parameters and strategies of style fusion according to the evaluation results and feedback them to the said style fusion module for optimization.

[0107] The evaluation index setting unit sets the evaluation indexes including user satisfaction, area attention promotion rate and target preference matching degree. The user satisfaction is collected through a questionnaire survey. The area attention promotion rate compares the changes in daily area attention before and after stylization. The target preference matching degree is the degree of coincidence between the stylization effect and the actual preferences of tourists.

[0108] The effect evaluation unit evaluates the stylization scheme according to the above indexes. For example, the score of the user satisfaction survey after stylization is increased from 75 points before to 85 points. The area attention promotion rate is calculated as: (the average number of 250 tourists per day after stylization - the original 200 tourists) ÷ 200 × 100% = 25%. The target preference matching degree reaches 70%.

[0109] Based on the evaluation results, if the target preference matching degree fails to meet the expectation, the parameter adjustment unit can adjust the weights of style feature extraction in the style fusion module. For example, it can increase the weight of the romantic and dreamy style feature vector, retrain the deep learning network, optimize the stylization scheme, and then iterate through the evaluation unit repeatedly until all indicators reach satisfactory results.

[0110] The visualization display module includes:

[0111] A scheme display unit for displaying the stylization scheme, including the correspondence between the image acquisition area and the style set, and the specific stylization effects of the images;

[0112] An evaluation result display unit for displaying the evaluation results of the stylization scheme.

[0113] The scheme display unit displays the stylization scheme through a visualization interface. The correspondence between the image acquisition area and the style set is listed on the left side of the interface. For example, the core area for flower display corresponds to the romantic and dreamy style set; the specific stylization effects of each image are shown in the middle part, such as the stylization display of adding special effects of butterflies flying and a pink background to the flower images in a certain exhibition area.

[0114] The evaluation result display unit displays the evaluation results of the stylization scheme on the right side of the visualization interface. It shows the user satisfaction score in a bar chart, the improvement of regional attention over time in a line chart, and the proportions of various target preference matching degrees in a pie chart, such as the proportions of like, dislike, and neutral. At the same time, the specific values of the evaluation indicators and the comparative analysis are listed to facilitate the organizer to intuitively understand the stylization effect and timely adjust the layout and display strategy of the flower exhibition.

[0115] The same or similar reference numerals correspond to the same or similar components;

[0116] The terms used to describe the positional relationship in the drawings are only for illustrative purposes and should not be construed as a limitation of this patent;

[0117] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, rather than limitations on the implementation manners of the present invention. For those of ordinary skill in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation manners here. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present invention shall be included in the protection scope of the claims of the present invention.

Claims

1. A flower show image stylization system based on multi-sampling anti-aliasing, characterized in that: include: An image acquisition module is used to acquire a plurality of image data of the flower show site at different angles and different time periods, and automatically divide the flower show site into a plurality of image acquisition areas according to shooting parameters of the image data, wherein the plurality of image acquisition areas respectively include a plurality of images; An image preprocessing module, used for acquiring image properties of the multiple images respectively included in the multiple image acquisition areas, and performing multi-sampling anti-aliasing processing on the multiple images according to the image properties to improve image quality; A style feature extraction module, used to obtain style information of multiple target style images, and classify the multiple target styles according to the style information to obtain multiple style sets, the number of the style sets is the same as the number of the image acquisition areas; An image content analysis module is used to obtain the viewing information corresponding to the multiple images respectively included in the multiple image acquisition areas, and predict the viewing satisfaction of the flowers that need to be newly displayed or the flowers that need to be adjusted in the layout in different exhibition areas of the flower exhibition according to the viewing information; a style fusion module, for respectively acquiring a plurality of stylized video information of the plurality of image acquisition regions, and analyzing the plurality of stylized video information to obtain daily regional attention, regional viewing time, and target preference, wherein the target preference is used to represent each viewer's attitude toward the plurality of target styles, wherein attitudes include like, dislike, and neutral; and for determining a stylization scheme of the plurality of images according to the viewing satisfaction, the daily regional attention, the regional viewing time, and the target preference, wherein the stylization scheme includes a correspondence between the plurality of image acquisition regions and the plurality of style sets, and specific stylization effects of the plurality of images; An effect evaluation module is used to evaluate the actual effect of the stylization scheme and adjust the parameters and strategies of style fusion according to the evaluation results; The visual display module is used to display the stylization scheme and evaluation results.

2. The flower show image stylization system according to claim 1, characterized in that: The image acquisition module comprises: The image data acquisition unit collects image data from different angles and at different times of the flower show site by combining drones and fixed cameras; The area division unit specifically performs cluster analysis on the shooting parameters of the image data, classifies the image data, and determines the optimal number of clusters through the cluster analysis using a silhouette coefficient to obtain the multiple image acquisition areas, wherein the number of the multiple image acquisition areas is the same as the optimal number of clusters.

3. The flower show image stylization system according to claim 2, characterized in that: The image preprocessing module comprises: An image attribute acquisition unit, used to acquire image attributes of the plurality of images, wherein the image attributes include resolution, color mode and edge complexity; The anti-aliasing processing unit dynamically adjusts the distribution and number of sampling points according to the edge complexity and resolution of the image, and uses an adaptive multi-sampling algorithm to perform anti-aliasing processing on the image.

4. The flower show image stylization system according to claim 3, characterized in that: The style feature extraction module includes: A style information acquisition unit, used to respectively acquire style information of the plurality of target style images by using big data technology, wherein the style information is text and image feature information; A style feature extraction unit, configured to extract features of the target style image according to the style information by using a feature extraction algorithm, and obtain style feature vectors corresponding to the multiple target styles, wherein the style feature vectors are used to represent features of each of the target styles; The style set determination unit performs cluster analysis on the style feature vectors corresponding to the multiple target styles respectively, classifies the multiple target styles according to the optimal number of clusters, and obtains the multiple style sets, wherein the number of the multiple style sets is the same as the optimal number of clusters.

5. The flower show image stylization system according to claim 4, characterized in that: The image content analysis module comprises: A viewing information acquisition unit, configured to acquire viewing information corresponding to the plurality of images respectively through big data technology, wherein the viewing information includes the number of viewers, the stay time, and evaluation feedback; A satisfaction degree calculation unit, configured to calculate viewing satisfaction degrees respectively corresponding to the plurality of images, wherein the plurality of images respectively correspond to one viewing satisfaction degree; The satisfaction prediction unit is used to train a deep learning network model according to the style feature vectors respectively corresponding to the multiple images, the location information of the images in the exhibition area and the viewing satisfaction, so as to obtain the satisfaction prediction model that can output the viewing satisfaction of the flowers that need to be newly displayed or the flowers that need to adjust the layout in different exhibition areas of the flower show; the new style feature vectors and location information corresponding to the newly added flowers or the flowers that need to adjust the layout are used as inputs of the satisfaction prediction model, and the satisfaction prediction model takes the viewing satisfaction of the newly added flowers or the flowers that need to adjust the layout as output.

6. The flower show image stylization system according to claim 1, characterized in that: The style fusion module includes: a stylized video information acquisition unit, configured to respectively acquire the plurality of stylized video information of the plurality of image acquisition areas; and further configured to respectively perform frame processing on the plurality of stylized video information to obtain a plurality of framed images, wherein the plurality of framed images respectively correspond to a time point, and the plurality of stylized video information of the plurality of image acquisition areas respectively include the plurality of framed images; The regional attention determination unit performs face recognition and headcount counting on the multiple sub-frame images based on image recognition technology to obtain the number of viewers appearing in the multiple sub-frame images, and obtains the number of viewers corresponding to the multiple sub-frame images according to the number of viewers; calculates the number of daily viewers corresponding to the multiple stylized video information of the multiple image acquisition areas according to the number of viewers corresponding to the multiple sub-frame images; performs mean calculation on the number of daily viewers corresponding to the multiple stylized video information to obtain the daily average number of viewers corresponding to the multiple image acquisition areas, and determines the daily average number of viewers as the daily regional attention; The area viewing time determination unit specifically performs face recognition on the multiple frame images respectively based on the image recognition technology, and determines the maximum time duration of each viewer appearing in the multiple frame images of the multiple image acquisition areas at multiple consecutive time points as the viewing time of each viewer in the multiple image acquisition areas; determines the corresponding time point when each viewer first appears in the multiple frame images of the multiple image acquisition areas as the first time point, determines the corresponding time point when each viewer last appears in the multiple frame images of the multiple image acquisition areas as the second time point, calculates the time difference between the first time point and the second time point, and determines the time difference as the maximum time; The target preference determination unit analyzes and processes the multiple sub-frame images respectively through an object detection algorithm to obtain the style positions of the target styles in the multiple sub-frame images respectively; then identifies the gaze direction of the viewer in the multiple sub-frame images through a gaze direction recognition model; and determines the target style viewed by the viewer in the multiple sub-frame images according to the style position and the gaze direction; finally, identifies the target preference feature vector of the viewer in the multiple sub-frame images through a preference recognition model, determines the preference corresponding to the target preference feature vector as the target preference, and associates the target preference with the target style; The stylization scheme determining unit is used to determine the stylization schemes of the plurality of images according to the viewing satisfaction, the daily regional attention, the regional viewing time and the target preference.

7. The flower show image stylization system according to claim 6, characterized in that: The gaze direction recognition model is specifically based on a plurality of first training images with actual gaze direction labels, and a deep learning network is trained to obtain the gaze direction recognition model that can output the gaze directions of the viewers appearing in the plurality of framed images; the preference recognition model is specifically based on a plurality of second training images with preference feature vectors, and a deep learning network is trained to obtain the preference recognition model that can output the preferences of the viewers appearing in the plurality of framed images; each preference feature vector corresponds to a preference label.

8. The flower show image stylization system according to claim 7, characterized in that: The stylization scheme determination unit includes: The comprehensive preference determination subunit specifically performs mean pooling on the multiple target preference feature vectors corresponding to the multiple target preferences to obtain a mean vector, performs similarity calculations on the mean vector and the multiple preference feature vectors, and determines the preference label corresponding to the preference feature vector having the highest similarity to the mean vector as the comprehensive preference; The corresponding relationship determination subunit, the regional feature vectors that respectively constitute the multiple image acquisition areas, normalize the daily regional attention and the regional viewing time; then use the graph attention network to assign weights to the normalized daily regional attention and the regional viewing time; finally, according to the weights corresponding to the daily regional attention and the regional viewing time, weight calculation is performed to obtain the regional feature vectors corresponding to the multiple image acquisition areas; the image feature vectors that respectively constitute the multiple images, the viewing satisfaction, and the mean vector of the comprehensive preference are normalized; then through the graph attention network The force network performs weight allocation on the normalized viewing satisfaction and the mean vector; finally, weight calculation is performed according to the weights corresponding to the viewing satisfaction and the mean vector, respectively, to obtain the image feature vectors corresponding to the multiple images; the corresponding relationship is determined, and similarity calculation is performed on each of the regional feature vectors and the multiple set feature vectors, respectively, to obtain the similarity between each of the regional feature vectors and each of the set feature vectors, the regional feature vector with the highest similarity is corresponded to the set feature vector, and the image acquisition area corresponding to the regional feature vector is corresponded to the style set corresponding to the set feature vector; The stylized effect determination subunit divides the multiple image acquisition areas into multiple stylized levels, and the multiple stylized levels are sorted in order of significance from high to low; then calculates the modulus lengths of the image feature vectors corresponding to the multiple images, and classifies them according to the modulus lengths of the image feature vectors to obtain multiple image stylized combinations, and the number of the image stylized combinations is the same as the number of the stylized levels; and calculates the total modulus length of the multiple image stylized combinations, and the multiple image stylized combinations are sorted in order from large to small according to the total modulus length. Finally, the correspondence between the multiple image stylized combinations and the multiple stylized levels is determined, and the corresponding rule is to correspond the image stylized combination corresponding to the largest total modulus length to the highest stylized level, and to correspond the remaining image stylized combinations to the remaining stylized levels in turn according to the sorting of the image stylized combinations and the sorting of the stylized levels.

9. The flower show image stylization system according to claim 1, characterized in that: The effect evaluation module includes: An evaluation index setting unit, used to set an index for evaluating the effect of a stylization scheme; An effect evaluation unit, used to evaluate the actual effect of the stylization scheme according to the evaluation index; The parameter adjustment unit is used to adjust the parameters and strategies of style fusion according to the evaluation results, and feed back to the style fusion module for optimization.

10. The flower show image stylization system according to claim 1, characterized in that: The visual display module includes: A scheme display unit, used to display the stylization scheme, including the correspondence between the image acquisition area and the style set, and the specific stylization effect of the image; The evaluation result display unit is used to display the evaluation result of the stylization scheme.

Images