Landscape plant image semantic segmentation method based on weak supervision

Abstract

The invention relates to a landscape plant image semantic segmentation method based on weak supervision. The method comprises the following steps: acquiring an input landscape plant image; preprocessing the input landscape plant image, and adaptively adjusting preprocessing parameters; extracting high-dimensional features of the landscape plant from the preprocessed landscape plant image by using a deep learning model; performing weak supervised learning training on the deep learning model through the partially labeled data; performing preliminary semantic segmentation on the input landscape plant image by using the trained deep learning model and the extracted high-dimensional features; performing post-processing on the preliminary segmentation result, and adaptively adjusting post-processing parameters; and outputting a landscape plant image after semantic segmentation according to a post-processing result. Compared with the prior art, the method has the advantages that identification and segmentation of landscape plants are specially optimized, adaptive adjustment and optimization processing are carried out on segmentation results, and edge details and visual consistency of the segmentation results are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of semantic segmentation, and in particular to a semantic segmentation method of landscape plant images based on weak supervision. Background Art

[0002] In landscape design and garden management, the identification and classification of landscape plants is a very important task. Traditional landscape plant identification methods usually rely on manual labeling and manual classification, which is not only time-consuming and laborious, but also requires high professional knowledge of the labelers. Therefore, how to efficiently realize the automatic identification and classification of landscape plants has become an urgent problem to be solved in this field.

[0003] With the development of computer vision technology, image semantic segmentation has become an effective means to solve this problem. Semantic segmentation is an advanced image processing task that aims to classify each pixel in an image into a specific category. However, existing image semantic segmentation methods usually rely on a large amount of labeled data for supervised learning training. These labeled data require fine annotation, high acquisition cost, and time-consuming and labor-intensive. For example, in large-scale landscape images, accurate pixel-level annotation of each plant not only requires a lot of time and human resources, but also requires the annotators to have high professional knowledge. This makes traditional supervised learning methods face great difficulties in practical applications.

[0004] In order to reduce the dependence on a large amount of labeled data, weakly supervised learning technology came into being. Weakly supervised learning can greatly reduce the cost and difficulty of data labeling by using partially labeled data for training. However, the existing weakly supervised learning methods still have some shortcomings when applied to landscape plant images in complex environments. First, there are many types of plants in landscape images with different shapes, and the same plants will perform differently in different environments. Existing methods are difficult to handle such highly complex and diverse scenes. Secondly, factors such as lighting changes, occlusion, and background interference will have a significant impact on the segmentation effect of the image. Existing methods have low robustness and accuracy when dealing with these complex environmental conditions.

[0005] In addition, most existing image semantic segmentation methods are used for the detection and segmentation of general objects, lacking special optimization for landscape plants. These methods usually ignore the biological characteristics and visual features of plants, and fail to make full use of the expertise of botany and landscape design, thus affecting the accuracy of the segmentation results. In addition, when existing methods process complex landscape images, the edge details and overall visual consistency of the segmentation results are often not ideal, which affects the actual application effect. Summary of the invention

[0006] The purpose of the present invention is to provide a semantic segmentation method for landscape plant images based on weak supervision, which reduces the dependence on a large amount of labeled data, improves the segmentation accuracy and robustness in complex environments, specifically optimizes the recognition and segmentation of landscape plants, and adaptively adjusts and optimizes the segmentation results, significantly improving the edge details and visual consistency of the segmentation results, and has significant practical value and broad application prospects.

[0007] The purpose of the present invention can be achieved by the following technical solutions:

[0008] A semantic segmentation method for landscape plant images based on weak supervision includes the following steps:

[0009] S1, obtaining an input landscape plant image;

[0010] S2, preprocessing the input landscape plant image, and adaptively adjusting the preprocessing parameters according to different landscape plant image qualities and environmental conditions;

[0011] S3, using a deep learning model to extract high-dimensional features of landscape plants from preprocessed landscape plant images;

[0012] S4, weakly supervised learning training of deep learning models through partially annotated data;

[0013] S5. Perform preliminary semantic segmentation on the input landscape plant image using the trained deep learning model and the extracted high-dimensional features, and classify each pixel in the landscape plant image into a specific landscape plant category;

[0014] S6, post-processing the preliminary segmentation results, and adaptively adjusting the post-processing parameters according to different landscape plant image qualities and environmental conditions;

[0015] S7. Outputting the semantically segmented landscape plant image based on the post-processing result.

[0016] The input forms of the landscape plant images include landscape plant images taken by a digital camera and landscape plant images extracted from an image database.

[0017] The preprocessing includes image enhancement, noise removal and scale transformation.

[0018] The pre-processing comprises the following steps:

[0019] S21, converting the acquired landscape plant images into different formats, converting the landscape plant images of different sources and formats into image formats supported by the processing system, the image formats including JPEG, PNG and TIFF formats;

[0020] S22, performing image resolution, image clarity and color saturation detection on the landscape plant image;

[0021] S23. Use the image quality enhancement algorithm based on gradient detection to perform image enhancement processing on the low-quality landscape plant images to improve the contrast and edge clarity of the landscape plant images:

[0022]

[0023] Among them, I enhanced represents the enhanced landscape plant image, I original represents the original landscape plant image, α is the enhancement coefficient, Represents image gradient;

[0024] S24, using an adaptive median filtering algorithm to remove noise from the landscape plant image that has undergone quality inspection and enhancement processing:

[0025] I denoised (x, y) = median{I enhanced (i,j)|(i,j)∈N(x,y)};

[0026] Among them, I denoised (x, y) represents the pixel value of the landscape plant image after denoising, N(x, y) represents the neighborhood pixel set centered at (x, y), and median is the median operation;

[0027] S25. According to the resolution and size of the landscape plant image, the size and resolution of the landscape plant image are adjusted to meet the requirements of subsequent processing steps, and the adjustment standard is set to a fixed size or proportional scaling:

[0028] I resized = resize(I denoised ,target_size);

[0029] Among them, I resized Represents the resized landscape plant image, resize is the resizing function, and target_size represents the target size;

[0030] S26, performing color space conversion on the resized landscape plant image, converting the image from RGB color space to Lab color space:

[0031] I Lab =convert(I resized ,RGB2Lab);

[0032] Among them, I Labrepresents the converted landscape plant image, convert is the color space conversion function, and RGB2Lab represents the conversion from RGB to Lab.

[0033] The S3 comprises the following steps:

[0034] S31, inputting the preprocessed landscape plant image into a multi-scale fusion deep learning model, wherein the multi-scale fusion deep learning model includes multiple parallel convolutional neural network branches, each convolutional neural network branch processes landscape plant image features of different scales;

[0035] S32. In each convolutional neural network branch, a multi-scale convolution operation is performed on the features of the input landscape plant image to extract local features of different scales:

[0036]

[0037] in, represents the convolution result of the s-th scale branch position (i, j) in the first layer, represents the convolution kernel weight of the s-th scale branch in the first layer, represents the input feature map of position (i+m,j+n) in layer 1-1, b l,s represents the bias of the s-th scale branch in the first layer, k s It represents half of the convolution kernel size of the s-th scale branch. The size of the convolution kernel is usually an odd number, so k s Indicates the number of positions from the center to the border of the convolution kernel size.

[0038] S33. After each convolution layer, an adaptive activation function is applied to the result of the convolution operation. The adaptive activation function dynamically adjusts the parameters according to the statistical characteristics of the input features and adjusts the nonlinear expression ability:

[0039]

[0040] in, represents the activation result of the s-th scale branch position (i, j) in the l-th layer, is a dynamic adjustment parameter, σ represents the standard deviation of the input feature, ReLU is an adaptive activation function, and γ is a dynamic adjustment parameter used to adjust the influence of the standard deviation of the input feature on the activation result.

[0041] S34, fuse the activation results of all scale branches and obtain the fused feature map by weighted summation:

[0042]

[0043] in, represents the feature map after fusion of the lth layer, α s is the weight coefficient of the sth scale branch, S is the total number of scale branches;

[0044] S35. Perform global feature extraction on the fused feature map and use the attention mechanism to enhance the global context information:

[0045]

[0046] in, represents the feature map after the attention mechanism is applied to the lth layer, and Represent the feature vectors of query, key and value respectively, and softmax is the normalized activation function;

[0047] S36. Globally pool the feature map processed by multi-scale fusion and attention mechanism to obtain a fixed-length feature vector representation as a high-dimensional feature representation of landscape plants.

[0048] The S4 comprises the following steps:

[0049] S41, dividing the partially labeled landscape plant image data into a training set and a validation set, and samples of each category are distributed in both sets;

[0050] S42. Define a loss function specific to landscape plant images, including biological characteristic loss and visual characteristic loss. The biological characteristic loss combines the biological characteristics of the plant, and the visual characteristic loss combines the visual characteristics of the plant. The mathematical expression of the loss function is:

[0051]

[0052] Among them, L represents the total loss, L bio Indicates the loss of biological properties, L visual represents the visual feature loss, L unsup represents the unsupervised loss, L struct represents the structural similarity loss, y i represents the true label of the i-th labeled sample, Represents the predicted label of the i-th labeled sample, I i represents the original image of the i-th labeled sample, represents the predicted image of the i-th labeled sample, y j represents the pseudo label of the jth unlabeled sample, represents the predicted label of the jth unlabeled sample, I j represents the original image of the jth unlabeled sample, represents the predicted image of the jth unlabeled sample, β and δ are weight coefficients, λ is the weight coefficient of unsupervised loss, N is the number of labeled samples, and M is the number of unlabeled samples;

[0053] S43, initializing the parameters of the multi-scale fusion deep learning model, and optimizing the multi-scale fusion deep learning model by a stochastic gradient descent algorithm;

[0054] S44. Perform forward propagation calculation on the labeled samples, obtain the predicted labels and calculate the biological characteristic loss and visual feature loss:

[0055]

[0056]

[0057] Among them, C is the number of landscape plant categories, Indicates that the i-th sample belongs to the true label of category c, represents the predicted probability that the i-th sample belongs to category c, w c is the biological characteristic weight of category c, represents the pixel neighborhood, I i (p,q) and Respectively represent the original and predicted values ​​of the i-th sample at the pixel point (p, q), G represents the visual feature similarity function, and μ is the weight coefficient of the visual feature similarity;

[0058] S45. Perform forward propagation calculation on unlabeled samples, generate pseudo labels through the model and calculate unsupervised loss and structural similarity loss:

[0059]

[0060]

[0061] Where H and W represent the height and width of the image respectively. and Respectively represent the original and predicted gradients of the jth sample at the pixel point (p, q), Represents the pseudo label of the jth unlabeled sample:

[0062]

[0063] S46, performing weighted summation of biological characteristic loss, visual feature loss, unsupervised loss and structural similarity loss to obtain a total loss, and updating the parameters of the multi-scale fusion deep learning model through a back propagation algorithm;

[0064] S47, repeat the forward propagation and back propagation process until the loss function on the validation set converges, and finally obtain the multi-scale fusion deep learning model f trained by weakly supervised learning θ* (x); where f θ (x) represents the multi-scale fusion deep learning model, x represents the input landscape plant image, θ * Represents the optimal parameter set obtained after weakly supervised learning training.

[0065] The S5 comprises the following steps:

[0066] S51, inputting the preprocessed landscape plant images and the extracted high-dimensional features into a multi-scale fusion deep learning model trained by weakly supervised learning;

[0067] S52, perform forward propagation calculation on the input landscape plant image to obtain the classification prediction probability of each pixel:

[0068]

[0069] in, represents the pixel classification prediction probability at position (i, j), f θ* (x i,j ) represents the pixel feature x of the multi-scale fusion deep learning model at position (i, j) i,j The output of , softmax is the normalized activation function;

[0070] S53, perform a maximum operation on the classification prediction probability of each pixel to determine the final classification label of each pixel:

[0071]

[0072] in, Represents the final classification label of the pixel at position (i, j), Indicates the predicted probability that the pixel at position (i, j) belongs to category c;

[0073] S54, combining the final classification labels of all pixels into a complete semantic segmentation map, indicating the plant category to which each pixel in the landscape plant image belongs.

[0074] The post-processing includes edge smoothing, region merging and noise filtering.

[0075] The edge smoothing is specifically as follows: edge detection and refinement processing are performed on the semantic segmentation map, and the segmentation result is optimized using a convolution post-processing algorithm to adjust the accuracy of the edge and the clarity of the details:

[0076]

[0077] in, Represents the optimized semantic segmentation result, conv is the convolution operation, They represent the semantic segmentation results of the adjacent pixels at position (i, j).

[0078] According to different landscape plant image quality and environmental conditions, the specific method of adaptively adjusting pre-processing or post-processing parameters is as follows:

[0079] According to the quality of the input landscape plant image and the environmental conditions, the adaptive adjustment parameter set θ = {θ1, θ2, …, θ n}, where each parameter θ i Corresponding to a specific pre-processing or post-processing operation, n is the number of parameters that need to be adjusted;

[0080] The quality of the input landscape plant image is evaluated by calculating the image contrast, brightness, noise level and resolution. The mathematical expression of the quality evaluation is:

[0081] Q = {q1,q2,q3,q4};

[0082] Wherein, q1 represents the contrast evaluation value, q2 represents the brightness evaluation value, q3 represents the noise level evaluation value, and q4 represents the resolution evaluation value;

[0083] According to the image quality assessment results and the preset threshold range, select the corresponding preprocessing parameter θ pre and post-processing parameters θ post ;

[0084] Apply the selected preprocessing parameters θ pre and post-processing parameters θ post Adaptive pre-processing and post-processing operations are performed separately.

[0085] Compared with the prior art, the present invention has the following beneficial effects:

[0086] (1) The present invention uses weakly supervised learning technology and only requires partially labeled data for training, which greatly reduces the dependence on the full amount of labeled data. It also combines biological characteristics and visual features for optimization, which not only reduces the cost and time investment of data labeling, but also reduces the dependence on professional labelers.

[0087] (2) The present invention significantly improves the recognition and segmentation accuracy of landscape plants in complex environments by adopting a multi-scale fusion deep learning model and an adaptive activation function. The multi-scale fusion deep learning model can effectively extract features of different scales and enhance the model's adaptability to plants of different sizes and shapes. In addition, the adaptive adjustment of the pre-processing and post-processing modules can dynamically adjust parameters according to image quality and environmental conditions, thereby ensuring the efficiency and accuracy of the processing steps and improving the robustness of the model under different environmental conditions.

[0088] (3) The present invention is optimized specifically for landscape plants, combining the expertise of botany and landscape design to improve the accuracy of plant species recognition. The visual consistency and practical application effect of the segmentation results are improved by post-processing the preliminary segmentation results, including edge smoothing, region merging and noise filtering. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] Figure 1 is a flow chart of the method of the present invention;

[0090] Figure 2 Schematic diagram of the structure of a multi-scale fusion deep learning model in an embodiment. DETAILED DESCRIPTION

[0091] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0092] This embodiment provides a landscape plant image semantic segmentation method based on weak supervision, such as Figure 1 As shown, the following steps are included:

[0093] S1. Obtain an input landscape plant image.

[0094] In this embodiment, the input form of the landscape plant image includes the landscape plant image taken by a digital camera and the landscape plant image extracted from the image database.

[0095] S2. Preprocessing the input landscape plant image, including image enhancement, noise removal and scale transformation, and adaptively adjusting the preprocessing parameters according to different landscape plant image qualities and environmental conditions.

[0096] In one embodiment, the preprocessing comprises the following steps:

[0097] S21, converting the acquired landscape plant images into different formats, converting the landscape plant images of different sources and formats into image formats supported by the processing system, the image formats including JPEG, PNG and TIFF formats;

[0098] S22, performing image resolution, image clarity and color saturation detection on the landscape plant image;

[0099] S23. Use the image quality enhancement algorithm based on gradient detection to perform image enhancement processing on the low-quality landscape plant images to improve the contrast and edge clarity of the landscape plant images:

[0100]

[0101] Among them, I enhanced represents the enhanced landscape plant image, I original represents the original landscape plant image, α is the enhancement coefficient, Represents image gradient;

[0102] S24, using an adaptive median filtering algorithm to remove noise from the landscape plant image that has undergone quality inspection and enhancement processing:

[0103] I denoised (x,y)=median{I enhanced (i,j)|(i,j)∈N(x,y)};

[0104] Among them, I denoised (x, y) represents the pixel value of the landscape plant image after denoising, N(x, y) represents the neighborhood pixel set centered at (x, y), and median is the median operation;

[0105] S25. According to the resolution and size of the landscape plant image, the size and resolution of the landscape plant image are adjusted to meet the requirements of subsequent processing steps, and the adjustment standard is set to a fixed size or proportional scaling:

[0106] I resized = resize(I denoised ,target_size);

[0107] Among them, I resized Represents the resized landscape plant image, resize is the resizing function, and target_size represents the target size;

[0108] S26, performing color space conversion on the resized landscape plant image, converting the image from RGB color space to Lab color space:

[0109] I Lab =convert(I resized ,RGB2Lab);

[0110] Among them, I Labrepresents the converted landscape plant image, convert is the color space conversion function, and RGB2Lab represents the conversion from RGB to Lab.

[0111] S3. Use deep learning models to extract high-dimensional features of landscape plants from preprocessed landscape plant images.

[0112] In one embodiment, S3 includes the following steps:

[0113] S31, inputting the preprocessed landscape plant image into a multi-scale fusion deep learning model, wherein the multi-scale fusion deep learning model includes multiple parallel convolutional neural network branches, each convolutional neural network branch processes landscape plant image features of different scales, such as Figure 2 As shown;

[0114] S32. In each convolutional neural network branch, a multi-scale convolution operation is performed on the features of the input landscape plant image to extract local features of different scales:

[0115]

[0116] in, represents the convolution result of the s-th scale branch position (i, j) in the l-th layer, represents the convolution kernel weight of the s-th scale branch of the l-th layer, represents the input feature map of position (i+m,j+n) in layer l-1, b l , s represents the bias of the s-th scale branch of the l-th layer, k s represents half of the convolution kernel size of the s-th scale branch. The size of the convolution kernel is usually an odd number, so k s Indicates the number of positions from the center to the border of the convolution kernel size.

[0117] S33. After each convolution layer, an adaptive activation function is applied to the result of the convolution operation. The adaptive activation function dynamically adjusts the parameters according to the statistical characteristics of the input features and adjusts the nonlinear expression ability:

[0118]

[0119] in, represents the activation result of the s-th scale branch position (i, j) in the l-th layer, is a dynamic adjustment parameter, σ represents the standard deviation of the input feature, ReLU is an adaptive activation function, and γ is a dynamic adjustment parameter used to adjust the influence of the standard deviation of the input feature on the activation result.

[0120] S34, fuse the activation results of all scale branches and obtain the fused feature map by weighted summation:

[0121]

[0122] in, represents the feature map after fusion of the lth layer, α s is the weight coefficient of the sth scale branch, S is the total number of scale branches;

[0123] S35. Perform global feature extraction on the fused feature map and use the attention mechanism to enhance the global context information:

[0124]

[0125] in, represents the feature map after the attention mechanism is applied to the lth layer, and Represent the feature vectors of query, key and value respectively, and softmax is the normalized activation function;

[0126] S36. Globally pool the feature map processed by multi-scale fusion and attention mechanism to obtain a fixed-length feature vector representation as a high-dimensional feature representation of landscape plants.

[0127] S4. Perform weakly supervised learning training on the deep learning model through partially labeled data.

[0128] In one embodiment, S4 includes the following steps:

[0129] S41, dividing the partially labeled landscape plant image data into a training set and a validation set, and samples of each category are distributed in both sets;

[0130] S42. Define a loss function specific to landscape plant images, including biological characteristic loss and visual characteristic loss. The biological characteristic loss combines the biological characteristics of the plant, and the visual characteristic loss combines the visual characteristics of the plant. The mathematical expression of the loss function is:

[0131]

[0132] Among them, L represents the total loss, L bio Indicates the loss of biological properties, L visual represents the visual feature loss, L unsup represents the unsupervised loss, L struct represents the structural similarity loss, y i represents the true label of the i-th labeled sample, Represents the predicted label of the i-th labeled sample, I irepresents the original image of the i-th labeled sample, represents the predicted image of the i-th labeled sample, y j represents the pseudo label of the jth unlabeled sample, represents the predicted label of the jth unlabeled sample, I j represents the original image of the jth unlabeled sample, represents the predicted image of the jth unlabeled sample, β and δ are weight coefficients, λ is the weight coefficient of unsupervised loss, N is the number of labeled samples, and M is the number of unlabeled samples;

[0133] S43, initializing the parameters of the multi-scale fusion deep learning model, and optimizing the multi-scale fusion deep learning model by a stochastic gradient descent algorithm;

[0134] S44. Perform forward propagation calculation on the labeled samples, obtain the predicted labels and calculate the biological characteristic loss and visual feature loss:

[0135]

[0136]

[0137] Among them, C is the number of landscape plant categories, Indicates that the i-th sample belongs to the true label of category c, represents the predicted probability that the i-th sample belongs to category c, w c is the biological characteristic weight of category c, represents the pixel neighborhood, I i (p,q) and Respectively represent the original and predicted values ​​of the i-th sample at the pixel point (p, q), G represents the visual feature similarity function, and μ is the weight coefficient of the visual feature similarity;

[0138] S45. Perform forward propagation calculation on unlabeled samples, generate pseudo labels through the model and calculate unsupervised loss and structural similarity loss:

[0139]

[0140]

[0141] Where H and W represent the height and width of the image respectively. and Respectively represent the original and predicted gradients of the jth sample at the pixel point (p, q), Represents the pseudo label of the jth unlabeled sample:

[0142]

[0143] S46, performing weighted summation of biological characteristic loss, visual feature loss, unsupervised loss and structural similarity loss to obtain a total loss, and updating the parameters of the multi-scale fusion deep learning model through a back propagation algorithm;

[0144] S47, repeat the forward propagation and back propagation process until the loss function on the validation set converges, and finally obtain the multi-scale fusion deep learning model f trained by weakly supervised learning θ* (x); where f θ (x) represents the multi-scale fusion deep learning model, x represents the input landscape plant image, θ * Represents the optimal parameter set obtained after weakly supervised learning training.

[0145] S5. Use the trained deep learning model and the extracted high-dimensional features to perform preliminary semantic segmentation on the input landscape plant image and classify each pixel in the landscape plant image into a specific landscape plant category.

[0146] In one embodiment, S5 includes the following steps:

[0147] S51, inputting the preprocessed landscape plant images and the extracted high-dimensional features into a multi-scale fusion deep learning model trained by weakly supervised learning;

[0148] S52, perform forward propagation calculation on the input landscape plant image to obtain the classification prediction probability of each pixel:

[0149]

[0150] in, represents the pixel classification prediction probability at position (i, j), f θ* (x i,j ) represents the pixel feature x of the multi-scale fusion deep learning model at position (i, j) i,j The output of , softmax is the normalized activation function;

[0151] S53, perform a maximum operation on the classification prediction probability of each pixel to determine the final classification label of each pixel:

[0152]

[0153] in, Represents the final classification label of the pixel at position (i, j), Indicates the predicted probability that the pixel at position (i, j) belongs to category c;

[0154] S54, combining the final classification labels of all pixels into a complete semantic segmentation map, indicating the plant category to which each pixel in the landscape plant image belongs.

[0155] S6. Post-process the preliminary segmentation results, including edge smoothing, region merging and noise filtering, and adaptively adjust the post-processing parameters according to different landscape plant image qualities and environmental conditions.

[0156] Among them, edge smoothing is specifically: edge detection and refinement of the semantic segmentation map, using the convolution post-processing algorithm to optimize the segmentation result, and adjusting the accuracy of the edge and the clarity of the details:

[0157]

[0158] in, Represents the optimized semantic segmentation result, conv is the convolution operation, They represent the semantic segmentation results of the adjacent pixels at position (i, j).

[0159] The region merging and noise filtering processes are conventional settings in the art and will not be described in detail here to avoid blurring the purpose of this application.

[0160] S7. Outputting the semantically segmented landscape plant image based on the post-processing result.

[0161] Output the optimized semantic segmentation map, visualize the classification results of each pixel, and obtain the final semantic segmentation result of the landscape plant image.

[0162] In this embodiment, the specific method of adaptively adjusting the pre-processing or post-processing parameters according to different landscape plant image qualities and environmental conditions is as follows:

[0163] According to the quality of the input landscape plant image and the environmental conditions, the adaptive adjustment parameter set θ = {θ1, θ2, …, θ n}, where each parameter θ i Corresponding to a specific pre-processing or post-processing operation, n is the number of parameters that need to be adjusted;

[0164] The quality of the input landscape plant image is evaluated by calculating the image contrast, brightness, noise level and resolution. The mathematical expression of the quality evaluation is:

[0165] Q = {q1,q2,q3,q4};

[0166] Wherein, q1 represents the contrast evaluation value, q2 represents the brightness evaluation value, q3 represents the noise level evaluation value, and q4 represents the resolution evaluation value;

[0167] According to the image quality assessment results and the preset threshold range, select the corresponding preprocessing parameter θ pre and post-processing parameters θ post ;

[0168] Apply the selected preprocessing parameters θ pre and post-processing parameters θ post Adaptive pre-processing and post-processing operations are performed separately.

[0169] Throughout the processing, the pre-processing and post-processing parameters Θ are dynamically adjusted based on the image quality and environmental conditions evaluated in real time.

[0170] In the management process of a certain garden scenic area, garden designers and maintenance personnel need to accurately identify and classify the plants in the scenic area in order to better carry out landscape design, plant maintenance and scenic area management. However, the plants in the scenic area are of various species and different shapes, and the appearance of the plants will also change significantly under different seasons and lighting conditions. Traditional manual labeling and manual classification methods are not only time-consuming and laborious, but also require high professional knowledge and are difficult to meet actual needs. Therefore, the landscape plant image semantic segmentation method based on weak supervision proposed in the present invention is applied to automatically identify and classify the plants in the scenic area.

[0171] In the process of managing the garden scenic area, this embodiment collects landscape plant image data in different seasons, different time periods and different lighting conditions. These data include high-resolution images taken by digital cameras and real-time monitoring images extracted from the scenic area monitoring system. In order to verify the effectiveness of the method of the present invention, we divide these image data into training sets and test sets, in which some images are accurately annotated for weakly supervised learning training, and the remaining images are used for model testing and verification.

[0172] In the image processing process, the input landscape plant image is first preprocessed. The preprocessing steps include image enhancement, noise removal, and scale transformation. Specifically, contrast enhancement, brightness adjustment, and color balance techniques are used to improve the visual quality of the image, while the adaptive median filter algorithm is used to remove noise from the image. The quality of the preprocessed image is significantly improved, laying a solid foundation for the subsequent feature extraction and segmentation steps.

[0173] Next, we use a multi-scale fusion deep learning model trained with weakly supervised learning to perform feature extraction and semantic segmentation on the preprocessed images. This model can extract plant features at different scales and enhance the expressiveness of features using adaptive activation functions and attention mechanisms. During the model training process, we use partially annotated data for supervised learning, and combine it with an unsupervised loss function to optimize the model parameters using a stochastic gradient descent algorithm, ultimately obtaining an efficient model trained with weakly supervised learning.

[0174] In practical applications, landscape plant images that have undergone preprocessing and feature extraction are input into the trained model for semantic segmentation. The model calculates the classification prediction probability of each pixel and determines the final classification label of each pixel based on the maximum value operation. The preliminary segmentation results are processed by the convolution post-processing algorithm for edge smoothing, region merging, and noise filtering, and finally an accurate semantic segmentation map is obtained.

[0175] In order to verify the effectiveness of the method of the present invention, multiple tests were conducted in the actual application of the garden scenic area, and the results were compared with the traditional manual annotation and manual classification methods. The test data includes 1,000 high-resolution landscape plant images in different seasons, different time periods and different lighting conditions. The following Table 1 is a data comparison of the test results:

[0176] Table 1 Comparison data of semantic segmentation methods for landscape plant images

[0177]

[0178] The data in Table 1 show that although the traditional manual labeling and manual classification methods can achieve a certain accuracy, they have a long processing time, low efficiency, and high reliance on professional knowledge. The method of the present invention only uses part of the labeled data and significantly reduces the cost and difficulty of data labeling through weakly supervised learning technology. In terms of processing time, the average processing time per image of the method of the present invention is only 5 minutes, which greatly improves the processing efficiency. In addition, the method of the present invention is superior to the traditional method in segmentation accuracy, edge details and user satisfaction, especially when processing landscape plant images in complex environments, it shows higher robustness and accuracy.

[0179] Through practical application and testing in the garden scenic area, this embodiment fully verifies the effectiveness and advantages of the method of the present invention. The method of the present invention not only solves the shortcomings of traditional methods in data annotation and processing efficiency, but also significantly improves the segmentation accuracy and visual consistency of landscape plant images, providing strong technical support for the design and management of garden scenic areas.

[0180] By using weakly supervised learning technology, the present invention only requires partially labeled data for training, which greatly reduces the dependence on the full amount of labeled data, and combines biological characteristics and visual features for optimization, which not only reduces the cost and time investment of data labeling, but also reduces the dependence on professional labelers.

[0181] The present invention significantly improves the recognition and segmentation accuracy of landscape plants in complex environments by adopting a multi-scale fusion deep learning model and an adaptive activation function. The multi-scale fusion deep learning model can effectively extract features of different scales and enhance the adaptability of the model to plants of different sizes and shapes. In addition, the adaptive adjustment of the pre-processing and post-processing modules can dynamically adjust parameters according to image quality and environmental conditions, ensuring the efficiency and accuracy of the processing steps and improving the robustness of the model under different environmental conditions.

[0182] The present invention is specially optimized for landscape plants, combining the expertise of botany and landscape design to improve the accuracy of plant species recognition. The visual consistency and practical application effect of the segmentation results are improved by post-processing the preliminary segmentation results, including edge smoothing, region merging and noise filtering.

[0183] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning, or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A semantic segmentation method for landscape plant images based on weak supervision, It is characterized in that The following steps are involved: S1, obtaining an input landscape plant image; S2, preprocessing the input landscape plant image, and adaptively adjusting the preprocessing parameters according to different landscape plant image qualities and environmental conditions; S3, using a deep learning model to extract high-dimensional features of landscape plants from preprocessed landscape plant images; S4, weakly supervised learning training of deep learning models through partially annotated data; S5. Perform preliminary semantic segmentation on the input landscape plant image using the trained deep learning model and the extracted high-dimensional features, and classify each pixel in the landscape plant image into a specific landscape plant category; S6, post-processing the preliminary segmentation results, and adaptively adjusting the post-processing parameters according to different landscape plant image qualities and environmental conditions; S7. Outputting the semantically segmented landscape plant image based on the post-processing result.

2. According to the weakly supervised semantic segmentation method of landscape plant images according to claim 1, It is characterized in that The input forms of the landscape plant images include landscape plant images taken by a digital camera and landscape plant images extracted from an image database.

3. According to the weakly supervised semantic segmentation method of landscape plant images according to claim 1, It is characterized in that The preprocessing includes image enhancement, noise removal and scale transformation.

4. According to the weakly supervised semantic segmentation method of landscape plant images according to claim 3, It is characterized in that The pre-processing comprises the following steps: S21, converting the acquired landscape plant images into different formats, converting the landscape plant images of different sources and formats into image formats supported by the processing system, the image formats including JPEG, PNG and TIFF formats; S22, performing image resolution, image clarity and color saturation detection on the landscape plant image; S23. Use the image quality enhancement algorithm based on gradient detection to perform image enhancement processing on the low-quality landscape plant images to improve the contrast and edge clarity of the landscape plant images: Among them, I enhanced represents the enhanced landscape plant image, I original represents the original landscape plant image, α is the enhancement coefficient, Represents image gradient; S24, using an adaptive median filtering algorithm to remove noise from the landscape plant image that has undergone quality inspection and enhancement processing: I denoised (z,y)=median{I enhanced (i,j)∣(i,j)∈N(x,y)}; Among them, I denoised (x, y) represents the pixel value of the landscape plant image after denoising, N(x, y) represents the neighborhood pixel set centered at (x, y), and median is the median operation; S25. According to the resolution and size of the landscape plant image, the size and resolution of the landscape plant image are adjusted to meet the requirements of subsequent processing steps, and the adjustment standard is set to a fixed size or proportional scaling: I resized =resize(I denoised ,target_size); Among them, I resized Represents the resized landscape plant image, resize is the resize function, and target_size represents the target size; S26, performing color space conversion on the resized landscape plant image, converting the image from RGB color space to Lab color space: I Lab =convert(I resized ,RGB2Lab); Among them, I Lab represents the converted landscape plant image, convert is the color space conversion function, and RGB2Lab represents the conversion from RGB to Lab.

5. The method for semantic segmentation of landscape plant images based on weak supervision according to claim 1, It is characterized in that The S3 comprises the following steps: S31, inputting the preprocessed landscape plant image into a multi-scale fusion deep learning model, wherein the multi-scale fusion deep learning model includes multiple parallel convolutional neural network branches, each convolutional neural network branch processes landscape plant image features of different scales; S32. In each convolutional neural network branch, a multi-scale convolution operation is performed on the features of the input landscape plant image to extract local features of different scales: in, represents the convolution result of the s-th scale branch position (i, j) in the l-th layer, represents the convolution kernel weight of the s-th scale branch of the l-th layer, represents the input feature map of position (i+m,j+n) in layer l-1, b l,s represents the bias of the s-th scale branch of the l-th layer, k s Represents half of the convolution kernel size of the s-th scale branch; S33. After each convolution layer, an adaptive activation function is applied to the result of the convolution operation. The adaptive activation function dynamically adjusts the parameters according to the statistical characteristics of the input features and adjusts the nonlinear expression ability: in, represents the activation result of the s-th scale branch position (i, j) in the l-th layer, is a dynamic adjustment parameter, σ represents the standard deviation of the input feature, ReLU is an adaptive activation function, and γ is a dynamic adjustment parameter used to adjust the impact of the standard deviation of the input feature on the activation result; S34, fuse the activation results of all scale branches, and obtain the fused feature map by weighted summation: in, represents the feature map after fusion of the lth layer, α s is the weight coefficient of the sth scale branch, S is the total number of scale branches; S35. Perform global feature extraction on the fused feature map and use the attention mechanism to enhance the global context information: in, represents the feature map after the attention mechanism is applied to the lth layer, and Represent the feature vectors of query, key and value respectively, and softmax is the normalized activation function; S36. Globally pool the feature map processed by multi-scale fusion and attention mechanism to obtain a fixed-length feature vector representation as a high-dimensional feature representation of landscape plants.

6. The method for semantic segmentation of landscape plant images based on weak supervision according to claim 1, It is characterized in that The S4 comprises the following steps: S41, dividing the partially labeled landscape plant image data into a training set and a validation set, and samples of each category are distributed in both sets; S42. Define a loss function specific to landscape plant images, including biological characteristic loss and visual characteristic loss. The biological characteristic loss combines the biological characteristics of the plant, and the visual characteristic loss combines the visual characteristics of the plant. The mathematical expression of the loss function is: Among them, L represents the total loss, L bio Indicates the loss of biological properties, L visual represents the visual feature loss, L unsup represents the unsupervised loss, L struct represents the structural similarity loss, y i represents the true label of the i-th labeled sample, Represents the predicted label of the i-th labeled sample, I i represents the original image of the i-th labeled sample, Represents the predicted image of the i-th labeled sample, y j represents the pseudo label of the jth unlabeled sample, represents the predicted label of the jth unlabeled sample, I j represents the original image of the jth unlabeled sample, represents the predicted image of the jth unlabeled sample, β and δ are weight coefficients, λ is the weight coefficient of unsupervised loss, N is the number of labeled samples, and M is the number of unlabeled samples; S43, initializing the parameters of the multi-scale fusion deep learning model, and optimizing the multi-scale fusion deep learning model by a stochastic gradient descent algorithm; S44. Perform forward propagation calculation on the labeled samples, obtain the predicted labels and calculate the biological characteristic loss and visual feature loss: Among them, C is the number of landscape plant categories, Indicates that the i-th sample belongs to the true label of category c, represents the predicted probability that the i-th sample belongs to category c, w c is the biological characteristic weight of category c, N represents the pixel neighborhood, I i (p,q) and Respectively represent the original and predicted values ​​of the i-th sample at the pixel point (p, q), G represents the visual feature similarity function, and μ is the weight coefficient of the visual feature similarity; S45. Perform forward propagation calculation on unlabeled samples, generate pseudo labels through the model and calculate unsupervised loss and structural similarity loss: Where H and W represent the height and width of the image respectively. and Respectively represent the original and predicted gradients of the jth sample at the pixel point (p, q), Represents the pseudo label of the jth unlabeled sample: S46, performing weighted summation of biological characteristic loss, visual feature loss, unsupervised loss and structural similarity loss to obtain a total loss, and updating the parameters of the multi-scale fusion deep learning model through a back propagation algorithm; S47, repeat the forward propagation and back propagation process until the loss function on the validation set converges, and finally obtain a multi-scale fusion deep learning model trained by weakly supervised learning Among them, f θ (x) represents the multi-scale fusion deep learning model, x represents the input landscape plant image, θ * Represents the optimal parameter set obtained after weakly supervised learning training.

7. The method for semantic segmentation of landscape plant images based on weak supervision according to claim 1, It is characterized in that The S5 comprises the following steps: S51, inputting the preprocessed landscape plant images and the extracted high-dimensional features into a multi-scale fusion deep learning model trained by weakly supervised learning; S52, perform forward propagation calculation on the input landscape plant image to obtain the classification prediction probability of each pixel: in, represents the predicted probability of pixel classification at position (i, j), Represents the pixel feature x of the multi-scale fusion deep learning model at position (i, j) i,j The output of , softmax is the normalized activation function; S53, perform a maximum operation on the classification prediction probability of each pixel to determine the final classification label of each pixel: in, Represents the final classification label of the pixel at position (i, j), Indicates the predicted probability that the pixel at position (i, j) belongs to category c; S54, combining the final classification labels of all pixels into a complete semantic segmentation map, indicating the plant category to which each pixel in the landscape plant image belongs.

8. The method for semantic segmentation of landscape plant images based on weak supervision according to claim 1, It is characterized in that The post-processing includes edge smoothing, region merging and noise filtering.

9. The method for semantic segmentation of landscape plant images based on weak supervision according to claim 8, It is characterized in that The edge smoothing is specifically as follows: edge detection and refinement processing are performed on the semantic segmentation map, and the segmentation result is optimized using a convolution post-processing algorithm to adjust the accuracy of the edge and the clarity of the details: in, Represents the optimized semantic segmentation result, conv is the convolution operation, They represent the semantic segmentation results of the adjacent pixels at position (i, j).

10. The method for semantic segmentation of landscape plant images based on weak supervision according to claim 1, It is characterized in that According to different landscape plant image quality and environmental conditions, the specific method of adaptively adjusting pre-processing or post-processing parameters is as follows: According to the quality of the input landscape plant image and the environmental conditions, the adaptive adjustment parameter set Θ = {θ 1 ,θ 2 ,…,θ n }, where each parameter θ i Corresponding to a specific pre-processing or post-processing operation, n is the number of parameters that need to be adjusted; The quality of the input landscape plant image is evaluated by calculating the image contrast, brightness, noise level and resolution. The mathematical expression of the quality evaluation is: Q={q 1 ,q 2 ,q 3 ,q 4 }; Among them, q 1 represents the contrast evaluation value, q 2 represents the brightness evaluation value, q 3 represents the noise level assessment value, q 4 represents the resolution evaluation value; According to the image quality assessment results and the preset threshold range, select the corresponding preprocessing parameter θ pre and post-processing parameters θ post ; Apply the selected preprocessing parameter θ pre and post-processing parameters θ post Adaptive pre-processing and post-processing operations are performed separately.

Images