A plant image segmentation method based on attention mechanism and multi-scale feature fusion

Abstract

The application discloses a plant image segmentation method based on an attention mechanism and multi-scale feature fusion, which comprises the following steps: dividing a training set, a verification set and a test set, and performing data processing; extracting multi-scale features, including deep features and shallow features; adjusting the channel weight of the deep features, and performing up-sampling processing on the deep feature maps to obtain deep feature maps guided by up-sampling; adjusting the spatial distribution weight of the shallow features to obtain shallow feature maps with adjusted spatial distribution weight; performing multi-scale fusion on the deep feature maps guided by up-sampling and the shallow feature maps with adjusted spatial distribution weight to obtain feature maps after multi-scale fusion; model training to obtain a trained plant image segmentation model; and finally, verifying and testing the model. The application can better solve the plant image segmentation and recognition problem under a complex background, has strong robustness and high accuracy, and can provide visual support for plant phenotype extraction and growth potential prediction.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of image processing, and relates to a plant image segmentation method based on an attention mechanism and multi-scale feature fusion. BACKGROUND

[0002] The phenotype of a plant is the result of the joint action of its own gene expression and environmental factors, and is an important factor for determining quality and shape. Therefore, the detection and analysis of the phenotype of a plant are of great significance. Traditional plant phenotype research mainly relies on manual observation and measurement, and has the disadvantages of high time and labor costs. In recent years, with the rapid development of computer technology, plant phenotype analysis technology based on computer vision brings great convenience to researchers due to its non-destructive extraction of plant phenotype characteristics and its fast and efficient characteristics. In order to obtain accurate plant phenotype data and predict growth, segmentation and recognition of plants are the premise and basis of the work. SUMMARY

[0003] The purpose of the application is to provide a plant image segmentation method based on an attention mechanism and multi-scale feature fusion.

[0004] The technical scheme is as follows:

[0005] S1, dividing a plant image dataset into a training set, a validation set and a test set, and respectively labeling original images in the training set, the validation set and the test set with labels to generate label files of the training set, the validation set and the test set, the labels including plants and backgrounds; converting each label file into a label image to obtain label images of the training set, the validation set and the test set;

[0006] S2, inputting the original images in the training set into a feature extraction network VGG16, and performing convolution on the original images by five convolutional layers in the VGG16 from shallow to deep to extract shallow features and deep features, and obtaining shallow feature maps and deep feature maps;

[0007] S3, adjusting channel weights of the deep features based on a channel attention module: first, performing global average pooling on each deep feature extracted in step S2 in a spatial dimension to obtain a channel attention vector v h ; second, using two consecutive fully connected layers to capture the weight relationship between channels; and third, mapping the channel attention vector v h to an interval [0, 1], and multiplying the channel attention vector v h mapped to the interval [0, 1] with a deep feature map corresponding to step S2 in a channel dimension to adjust the channel weight of the corresponding deep feature map, to obtain a corresponding deep feature map with adjusted channel weights, and finally to obtain each deep feature map with adjusted channel weights;

[0008] S4, the adjusted channel weight of each deep feature map output by step S3 is first respectively up-sampled, the resolution is expanded using a linear interpolation function, and then each deep feature map after up-sampling guidance is obtained by using an up-sampling guidance module;

[0009] S5, adjusting the spatial distribution weight of each shallow feature based on a spatial attention module: first, the maximum pooling and average pooling are respectively performed on each shallow feature map obtained in step S2 in the channel dimension to obtain the maximum pooled shallow feature map and the average pooled shallow feature map; secondly, the maximum pooled shallow feature map and the average pooled shallow feature map corresponding thereto are spliced and convoluted to obtain each two-dimensional attention map, i.e. the spatial weight coefficient of each shallow feature; finally, the spatial weight coefficient of each shallow feature is mapped to the interval [0, 1] respectively, and each spatial weight coefficient mapped to the interval [0, 1] is multiplied by the corresponding shallow feature map in step S2 to adjust the weight of each part in the spatial latitude of the corresponding shallow feature map, thereby obtaining the shallow feature map after adjusting the spatial distribution weight, and finally obtaining the shallow feature map after adjusting the spatial distribution weight;

[0010] S6, multi-scale feature fusion: the up-sampling guided deep feature map obtained in step S4 and the shallow feature map after adjusting the spatial distribution weight obtained in step S5 are spliced in the channel dimension using a concatenate function to obtain a spliced feature map, and the spliced feature map is mapped to the interval [0, 1] to obtain a multi-scale feature fused feature map;

[0011] S7, the multi-scale feature fused feature map obtained in step S6 is used to calculate the cross-entropy loss value with the labeled map in the training set, the cross-entropy loss value is called backward function to update the parameters of each convolution layer of the neural network, and the process is repeated until the cross-entropy loss value tends to a stable value, thereby obtaining the segmentation model of the plant image, and the model is saved;

[0012] S8, the plant image segmentation model obtained in step S7 is verified by using the original image in the verification set, if the model is not good, the model training is re-performed; if the model is good, the plant image segmentation model is tested by using the original image in the test set to obtain the segmented plant binary image.

[0013] Further, the original images in the training set, the verification set and the test set are manually labeled by using the labelme image labeling software in step S1, and the label file is in json format; the labelme_json_to_dataset command is used to convert the label file in json format into the labeled map in jpg format.

[0014] Further, the convolution function of the convolution layer in step S2 is Conv2d, the first layer and the second layer of the convolution layer extract shallow features, and output the first shallow feature map and the second shallow feature map after convolution respectively, the third layer, the fourth layer and the fifth layer of the convolution layer extract deep features, and output the first deep feature map, the second deep feature map and the third deep feature map after convolution respectively; the shallow features and the deep features are features of different scales.

[0015] Further, the calculation formula of the deep feature map after adjusting the channel weight in step S3 is:

[0016] M c (F l )=F(v h )=σ(fc2(δ(fc1(v h ))))

[0017] Wherein, M c (F l ) represents the deep feature map after adjusting the channel weight, F l represents the deep feature map, σ represents the Sigmoid function, fc1 and fc2 represent two full connection layers respectively, δ represents the ReLU function, v h represents the channel attention vector, and F(v h ) represents a function of adjusting the weight of the channel attention vector.

[0018] Further, the upsampling guide module in step S4 is specifically: the deep features after upsampling are respectively down-sampled by 2 times, 4 times and 8 times, averaged-pooled, and then restored to the original size by 2 times, 4 times and 8 times of upsampling, and finally added to the deep features after upsampling and averaged.

[0019] Further, the calculation formula of the shallow feature map after adjusting the spatial distribution weight in step S5 is:

[0020] M s (F d )=σ(f 7×7 ([AvgPool(F d );MaxPool(F d )]))

[0021] =σ(f 7×7 ([F avg ;F max ]))

[0022] Wherein, M s (F d ) represents the shallow feature map after adjusting the spatial distribution weight, F d represents the shallow feature map, σ represents the Sigmoid function, f7×7 This represents a convolution operation with a kernel size of 7×7, F avg F represents the shallow feature map after average pooling. max This represents the shallow feature map after max pooling.

[0023] Furthermore, in step S7, the cross-entropy loss value is calculated using the loss function BCELoss().

[0024] The present invention provides a plant image segmentation system based on attention mechanism and multi-scale feature fusion, comprising:

[0025] The data processing module is used to divide the plant image dataset into training, validation, and test sets, and to label the original images in the training, validation, and test sets respectively, generating label files for the training, validation, and test sets. The labels include plants and background. The label files are then converted into labeled images to obtain labeled images for the training, validation, and test sets.

[0026] The feature extraction module is used to extract shallow and deep features from the original image and output shallow and deep feature maps.

[0027] The deep feature map update module is used to adjust the channel weights of deep features and output the deep feature map after adjusting the channel weights.

[0028] The shallow feature map update module is used to adjust the spatial distribution weights of shallow features and output the shallow feature map after adjusting the spatial distribution weights.

[0029] The multi-scale feature fusion module is used to concatenate the deep feature map after upsampling guidance and the shallow feature map after adjusting the spatial distribution weights using the concatenate function, and map the concatenated feature map to the [0,1] interval;

[0030] The training module is used to calculate the cross-entropy loss value by combining the spliced ​​feature map mapped to the [0,1] interval with the label image in the training set. The cross-entropy loss value is used to call the backward function to backpropagate and update the parameters of each convolutional layer of the neural network. This process is iterated repeatedly until the cross-entropy loss value tends to a stable value, thus obtaining the segmentation model of the plant image and saving the model.

[0031] The verification module is used to verify the saved model using the original images in the verification set.

[0032] The testing module is used to test the validated model using the original images in the test set, and obtain the segmented binary image of the plant.

[0033] An apparatus device of the present application comprises a memory and a processor, wherein the memory is used to store a computer program capable of running on the processor; the processor is used to execute the steps of the above-mentioned plant image segmentation method based on attention mechanism and multi-scale feature fusion when running the computer program.

[0034] A storage medium of the present application, the storage medium has a computer program stored thereon, the computer program realizes the steps of the above-mentioned plant image segmentation method based on attention mechanism and multi-scale feature fusion when executed by at least one processor.

[0035] Advantages: compared with the prior art, the technical effects of the present application are: (1) adopting the idea of image multi-scale feature fusion, the model comprehensively utilizes deep and shallow features through feature fusion, plays the advantages of multi-scale features, the features are more rich, and the model has stronger generalization degree; (2) the attention mechanism is introduced, the spatial attention module and the channel attention module are used to optimize deep and shallow feature information, the interference of useless information on the model is weakened, the influence of important information on the model is enhanced, so that the network learns more effective plant image features, and the segmentation precision is improved; (3) the precision reduction caused by up-sampling is considered, the up-sampling guiding module is introduced to relieve the aliasing effect caused by linear interpolation in the up-sampling operation, and the precision is further improved. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 is a method flowchart of the present application;

[0037] Figure 2 is a segmentation effect comparison diagram of the traditional method and the method of the present application, wherein (a) is an original image, (b) is a segmentation effect diagram using the K-means algorithm, and (c) is a segmentation effect diagram of the method of the present application. DETAILED DESCRIPTION

[0038] The present application will be described in detail below in combination with the drawings and specific embodiments.

[0039] This embodiment takes dwarf yellow cabbage as an example, the dwarf yellow cabbage is a fine cabbage variety originally from the suburbs of Nanjing, and has good edible quality and strong disease resistance. The phenotype of the dwarf yellow cabbage is the result of the joint action of its own gene expression and environmental factors, and is an important factor for determining quality and shape. Therefore, the detection and analysis of the phenotype of the dwarf yellow cabbage are of great significance.

[0040] As shown in Figure 1 A plant image segmentation method based on attention mechanism and multi-scale feature fusion of the present application comprises the following steps:

[0041] S1, divide the plant image dataset into a training set, a validation set and a test set, and label the original images in the training set, the validation set and the test set respectively, generate label files of the training set, the validation set and the test set, and the label includes plants and backgrounds; convert each label file into a label image to obtain the label images of the training set, the validation set and the test set;

[0042] In the embodiment of the application, the dwarf yellow cabbage image dataset is 3360 greenhouse dwarf yellow cabbage images. The shooting location is the Jiangsu Liuhai Modern Agricultural Industry Demonstration Park dwarf yellow cabbage greenhouse in Liuhai District, Nanjing. The 3360 dwarf yellow cabbage image dataset is divided into a training set, a validation set and a test set, and the number of original images in the three sample sets is 2688, 336 and 336 respectively, and the division ratio is 8:1:1. The labelme image labeling software is used to manually label all original images in the three sample sets to obtain label files of the training set, the validation set and the test set, and the labels are two types, namely dwarf yellow cabbage and background, and the label file is in json format; the labelme_json_to_dataset command is used to convert the json format label file into a jpg format label image to obtain the label images of the training set, the validation set and the test set.

[0043] S2, input the original image in the training set into the feature extraction network VGG16, and use the five convolutional layers from shallow to deep in VGG16 to convolve the original image to extract shallow features and deep features, the convolution function of the convolutional layer in step S2 is Conv2d, the first layer and the second layer extract shallow features and output the first shallow feature map and the second shallow feature map after the first second layer convolution, the third layer, the fourth layer and the fifth layer extract deep features and output the first deep feature map, the second deep feature map and the third deep feature map after the third fourth fifth layer convolution; the shallow features and the deep features are features of different scales, the shallow image has rich details but lacks semantic information, and the deep image is the opposite.

[0044] S3, adjust the channel weight of the deep feature based on the channel attention module, the channel attention module models the importance of different channels of the dwarf yellow cabbage image, gives a higher weight to the channel with a higher response to the dwarf yellow cabbage area, enhances the effective semantic information, and gives a smaller weight to the channel with a lower response, and suppresses useless information.

[0045] Specifically, input each deep feature output by the third fourth fifth layer convolutional layer in VGG16, first perform global average pooling (Average pooling) on the spatial dimension to obtain a channel attention vector v hSecondly, two consecutive fully connected layers (FC) are used to capture the inter-channel weight relationship, and finally the sigmiod function is used to obtain the channel attention vector v h to the interval [0, 1], and the channel attention vector v h after being mapped to the interval [0, 1] is multiplied in the channel dimension corresponding to the deep feature map in step S2, the channel weight of the corresponding deep feature map is adjusted, and the corresponding deep feature map after adjusting the channel weight is obtained; the formula is:

[0046] M c (F l )=F(v h )=σ(fc2(δ(fc1(v h ))))

[0047] Where M c (F l ) represents the deep feature map after adjusting the channel weight, F l represents the deep feature map, sigma represents the Sigmoid function, fc1 and fc2 represent two fully connected layers respectively, and delta represents the ReLU function. v h represents the channel attention vector, and F(v h ) represents a function of adjusting the weight of the channel attention vector.

[0048] Deep feature semantic information is rich, and the channel attention module CA is used to enhance the expression ability of the high response channel.

[0049] S4, the deep feature needs to be up-sampled, and the interpolation method will cause errors. In order to reduce the error, an up-sampling guide module is needed to further process the up-sampled deep feature; specifically: the deep feature map after adjusting the channel weight output by step S3 is first up-sampled, the resolution is expanded using a linear interpolation function, and then the up-sampling guide module is used for processing to reduce the error. The up-sampling guide module is specifically: each deep feature after up-sampling is down-sampled by 2, 4 and 8 times, averaged and then restored to the original size by 2, 4 and 8 times up-sampling, and finally added to the deep feature after up-sampling and averaged.

[0050] S5, the spatial distribution weight of each shallow feature is adjusted based on the spatial attention module. The spatial attention module processes the image in the spatial scale, focuses on the spatial position information of the short stem yellow mustard in the image, and gives it a larger weight, while giving the background and other non-important information a smaller weight to suppress.

[0051] Specifically, input the respective shallow feature maps output by the first layer and the second layer in the VGG16, first perform max pooling and average pooling on the channel dimension respectively to obtain the respective shallow feature maps after max pooling and the respective shallow feature maps after average pooling, and meanwhile, using the two pooling methods can extract more rich features and enhance the network expression capability; secondly, the respective shallow feature maps after max pooling and the respective shallow feature maps after average pooling corresponding thereto are spliced (Concat) and subjected to 7x7 convolution to obtain respective two-dimensional attention maps, i.e. spatial weight coefficients; finally, the sigmoid operation is used to map the respective spatial weight coefficients to the [0, 1] interval, and the respective spatial weight coefficients mapped to the [0, 1] interval are multiplied with the corresponding shallow feature maps in step S2 to adjust the weight of each part of the corresponding shallow feature map in the spatial latitude, and obtain the corresponding shallow feature map after adjusting the spatial distribution weight; the formula is:

[0052] M s (F d )=σ(f 7×7 ([AvgPool(F d );MaxPool(F d )]))

[0053] =σ(f 7×7 ([F avg ;F max ]))

[0054] Wherein, M s (F d ) represents the shallow feature map after adjusting the spatial distribution weight, F d represents the shallow feature map, sigma represents the Sigmoid function, f 7×7 represents the convolution operation with the convolution kernel size of 7x7, F avg represents the shallow feature map after average pooling, and F max represents the shallow feature map after max pooling.

[0055] The shallow feature contains more texture details, and the spatial attention module SA is used to capture more important spatial information.

[0056] S6, multi-scale feature fusion, the resolution of the shallow layer feature in the deep network is higher, the image texture detail information is more, but because the convolution is less, the semantic information is less and the noise is more, while the deep layer feature contains more rich semantic information and less noise, the outline of the object in the deep layer feature is more obvious, but the perception ability for image details is poor, and the resolution is low. Therefore, by using the feature fusion method, the deep layer feature is guided to the shallow layer feature to better locate the position and outline of the significant object, and the shallow layer feature provides position detail information, so that the neural network performance can be effectively improved.

[0057] The up-sampling guided deep layer feature maps obtained in step S4 and the shallow layer feature maps after adjusting the spatial distribution weight obtained in step S5 are spliced in the channel dimension by using a concatenate function to obtain a spliced feature map, and the spliced feature map is mapped to the interval [0, 1] by using a sigmoid activation function to obtain a feature map after multi-scale feature fusion.

[0058] S7, the cross-entropy loss is calculated by using the feature map after multi-scale feature fusion output by step S6 and the label map in the training set, the loss function is BCELoss(), the cross-entropy loss value is updated by calling the backward function to update the parameters of each convolution layer of the neural network, and the process is repeated until the cross-entropy loss value tends to a stable value, so that the plant image segmentation model is obtained, and the model is saved.

[0059] S8, the plant image segmentation model obtained in step S7 is verified by using the original image in the verification set, if the model is not good, the model training is re-performed, if the model is good, the plant image segmentation model after verification is loaded, the green vegetable original image in the test set is input, and the segmented green vegetable binary image is obtained.

[0060] The plant image segmentation system based on the attention mechanism and the multi-scale feature fusion includes:

[0061] The data processing module is used for dividing the plant image data set into a training set, a verification set and a test set, and labeling the original images in the training set, the verification set and the test set by using labels respectively, generating label files of the training set, the verification set and the test set, and the label includes plants and backgrounds; the label files are converted into label maps to obtain the label maps of the training set, the verification set and the test set;

[0062] The feature extraction module is used for extracting shallow layer features and deep layer features of the original image, and outputting shallow layer feature maps and deep layer feature maps;

[0063] The deep layer feature map updating module is used for adjusting the channel weight of the deep layer feature, and outputting the deep layer feature map after adjusting the channel weight;

[0064] The shallow feature map updating module is configured to adjust the spatial distribution weight of the shallow feature and output the shallow feature map after the spatial distribution weight is adjusted.

[0065] The multi-scale feature fusion module is configured to concatenate the deep feature map after the up-sampling guidance and the shallow feature map after the spatial distribution weight is adjusted by using a concatenate function, and map the concatenated feature map to the interval [0, 1].

[0066] The training module is configured to calculate the cross-entropy loss value of the concatenated feature map after being mapped to the interval [0, 1] and the label image in the training set, call the backward function to update the parameters of each convolutional layer of the neural network, and repeatedly iterate the process until the cross-entropy loss value tends to a stable value, so as to obtain the segmentation model of the plant image and save the model.

[0067] The verification module is configured to verify the saved model by using the original image in the verification set.

[0068] The test module is configured to test the verified model by using the original image in the test set, so as to obtain the segmented plant binary image.

[0069] The device of the application comprises a memory and a processor, wherein the memory is configured to store a computer program capable of running on the processor; the processor is configured to execute the steps of the above-mentioned plant image segmentation method based on the attention mechanism and multi-scale feature fusion when the computer program is running, and achieve the same technical effects as the above-mentioned method.

[0070] The storage medium of the application stores a computer program, and the computer program is executed by at least one processor to implement the steps of the above-mentioned plant image segmentation method based on the attention mechanism and multi-scale feature fusion, and achieve the same technical effects as the above-mentioned method.

[0071] In the application experiment of the plant image segmentation algorithm based on the attention mechanism and multi-scale feature fusion, a large number of actual scene green vegetable images are tested. Figure 2 The comparison chart of the segmentation effects of the traditional method and the method of the application is shown in Figure 1, wherein the traditional method uses the K-means algorithm to segment the green vegetable image, and the segmentation result is compared with the segmentation result of the application. Figure 2 As can be seen from (a) to (c) in Figure 1, the traditional method has low segmentation accuracy and cannot accurately identify the short stem yellow green vegetable area and the background area, while the segmentation accuracy of the application is high, the boundary after segmentation is clear, and the segmentation effect is excellent.

[0072] The method can identify early short foot yellow pixels and realize segmentation, on the basis of a pre-trained VGG16 network, channel attention modules and spatial attention modules are used to adjust deep features and shallow features respectively, multi-scale features are fused, an up-sampling guiding module is introduced to weaken the feature aliasing effect caused by up-sampling before fusion, and the segmentation and identification of plant images are completed under the supervision of a label image. The method can better solve the plant image segmentation and identification problem in a complex background, has strong robustness and high accuracy, and can provide visual support for plant phenotype extraction and growth prediction.

Claims

1. A plant image segmentation method based on attention mechanism and multi-scale feature fusion, characterized in that, The method comprises the following steps: S1, dividing a plant image dataset into a training set, a validation set and a test set, and labeling the original images in the training set, the validation set and the test set respectively using labels to generate label files of the training set, the validation set and the test set, wherein the labels comprise plants and backgrounds; and converting the label files into annotation images to obtain the annotation images of the training set, the validation set and the test set; S2, inputting the original images in the training set into a feature extraction network VGG16, and performing convolution on the original images by five convolutional layers in the VGG16 from shallow to deep to extract shallow features and deep features, and obtaining shallow feature maps and deep feature maps; S3, adjusting the channel weights of the deep features respectively based on the channel attention module: taking each deep feature extracted in step S2, performing global average pooling on the spatial dimension to obtain a channel attention vector v h ; secondly, using two consecutive fully connected layers to capture the weight relationship between channels; finally, mapping the channel attention vector v h to the interval [0, 1], and multiplying the deep feature map corresponding to the channel attention vector v h mapped to the interval [0, 1] with the deep feature map in step S2 in the channel dimension to adjust the channel weights of the corresponding deep feature map, obtaining the corresponding deep feature map after adjusting the channel weights, and finally obtaining each deep feature map after adjusting the channel weights; S4, first performing upsampling on each deep feature map output by step S3 respectively, using a linear interpolation function to expand the resolution, and then using an upsampling guide module to process to obtain each deep feature map after upsampling guidance; S5, adjusting the spatial distribution weight of each shallow feature based on a spatial attention module: first performing maximum pooling and average pooling on each shallow feature map obtained in step S2 in the channel dimension to obtain each shallow feature map after maximum pooling and each shallow feature map after average pooling; secondly, splicing and convolving the shallow feature map after maximum pooling and the shallow feature map after average pooling corresponding thereto to obtain each two-dimensional attention map, i.e., the spatial weight coefficient of each shallow feature; finally, mapping the spatial weight coefficient of each shallow feature to the interval [0, 1] respectively, and multiplying each spatial weight coefficient after being mapped to the interval [0, 1] with the corresponding shallow feature map in step S2 to adjust the weight of each part in the spatial latitude of the corresponding shallow feature map, to obtain the corresponding shallow feature map after adjusting the spatial distribution weight, and finally to obtain each shallow feature map after adjusting the spatial distribution weight; S6, multi-scale feature fusion: splicing the deep feature map after upsampling guidance obtained in step S4 and the shallow feature map after adjusting the spatial distribution weight obtained in step S5 in the channel dimension using a concatenate function to obtain a spliced feature map, and mapping the spliced feature map to the interval [0, 1] to obtain a feature map after multi-scale feature fusion; S7, calculating a cross-entropy loss value using the feature map after multi-scale feature fusion obtained in step S6 and the annotation image in the training set, calling a backward function to update the parameters of each convolutional layer of the neural network, and repeatedly iterating this process until the cross-entropy loss value tends to a stable value, obtaining a plant image segmentation model, and saving the model; S8, verifying the plant image segmentation model obtained in step S7 using the original images in the validation set, if the model is not good, retraining the model; if the model is good, testing the plant image segmentation model using the original images in the test set to obtain a segmented plant binary image.

2. The plant image segmentation method based on attention mechanism and multi-scale feature fusion according to claim 1, characterized in that, In step S1, the original images in the training set, the verification set and the test set are manually labeled by using the labelme image labeling software, and the label file is in json format; and the label file in json format is converted into the labeled image in jpg format by using the labelme_json_to_dataset command.

3. The plant image segmentation method based on attention mechanism and multi-scale feature fusion according to claim 1, characterized in that, In step S2, the convolution function of the convolution layer is Conv2d, the first layer and the second layer of the convolution layer extract shallow features, and output the first shallow feature map and the second shallow feature map after convolution respectively, the third layer, the fourth layer and the fifth layer of the convolution layer extract deep features, and output the first deep feature map, the second deep feature map and the third deep feature map after convolution respectively; the shallow features and the deep features are features of different scales.

4. The plant image segmentation method based on attention mechanism and multi-scale feature fusion according to claim 1, characterized in that, In step S3, the calculation formula of the deep feature map after adjusting the channel weight is: wherein M c (F l ) represents the deep feature map after adjusting the channel weight, F l represents the deep feature map, σ represents the Sigmoid function, fc1 and fc2 represent two fully connected layers respectively, δ represents the ReLU function, v h represents the channel attention vector, F(v h ) represents a function of adjusting the weight of the channel attention vector.

5. The plant image segmentation method based on attention mechanism and multi-scale feature fusion according to claim 1, characterized in that, In step S4, the upsampling guiding module is specifically: the deep features after upsampling are respectively down-sampled by 2 times, 4 times and 8 times, averaged-pooled, and then restored to the original size by upsampling by 2 times, 4 times and 8 times, and finally added to the deep features after upsampling and averaged.

6. The plant image segmentation method based on attention mechanism and multi-scale feature fusion according to claim 1, characterized in that, In step S5, the calculation formula of the shallow feature map after adjusting the spatial distribution weight is: M s (F d ) = σ(f 7×7 ([AvgPool(F d ) ; MaxPool(F d )])) = σ(f 7×7 ([F avg ; F max ])) wherein M s (F d ) denotes the shallow feature map after adjusting the spatial distribution weight, F d denotes the shallow feature map, and σ denotes the Sigmoid function, f 7×7 denotes the convolution operation with a convolution kernel size of 7x7, F avg denotes the shallow feature map after average pooling, F max denotes the shallow feature map after maximum pooling.

7. The plant image segmentation method based on attention mechanism and multi-scale feature fusion according to claim 1, characterized in that, In step S7, the loss function BCELoss() is used to calculate the cross-entropy loss value.

8. A system for the plant image segmentation method based on attention mechanism and multi-scale feature fusion according to any one of claims 1 to 7, characterized in that, The method comprises the following steps: A data processing module is configured to divide a plant image dataset into a training set, a verification set and a test set, and label the original images in the training set, the verification set and the test set by using labels respectively to generate label files of the training set, the verification set and the test set, wherein the labels include plants and backgrounds; and convert the label files into labeled images to obtain labeled images of the training set, the verification set and the test set. A feature extraction module is configured to extract shallow features and deep features of the original images and output shallow feature maps and deep feature maps. A deep feature map updating module is configured to adjust the channel weight of the deep features and output deep feature maps after adjusting the channel weight. A shallow feature map updating module is configured to adjust the spatial distribution weight of the shallow features and output shallow feature maps after adjusting the spatial distribution weight. A multi-scale feature fusion module is configured to concatenate the deep feature maps after upsampling and the shallow feature maps after adjusting the spatial distribution weight by using a concatenate function, and map the concatenated feature maps to the [0, 1] interval. A training module is configured to calculate a cross-entropy loss value between the concatenated feature maps mapped to the [0, 1] interval and the label images in the training set, call a backward function to update the parameters of each convolution layer of the neural network, and repeatedly iterate the process until the cross-entropy loss value tends to a stable value, so as to obtain a plant image segmentation model and save the model. A verification module is configured to verify the saved model by using the original images in the verification set. A test module is configured to test the verified model by using the original images in the test set to obtain a segmented plant binary image.

9. An apparatus device comprising: The system comprises a memory and a processor, wherein: The memory is configured to store a computer program capable of running on the processor. A processor configured to implement the steps of the plant image segmentation method based on attention mechanism and multi-scale feature fusion according to any one of claims 1-7 when executing the computer program.

10. A storage medium, characterized by A computer program stored on the storage medium, which, when executed by at least one processor, implements the steps of the plant image segmentation method based on attention mechanism and multi-scale feature fusion according to any one of claims 1-7.

Images