A method for assessing phloem damage in mango tree branches

Abstract

This application discloses a method for assessing damage to the phloem of mango tree branches. The method involves acquiring images of the phloem using a camera and employing AI-based image recognition and processing techniques to perform multi-scale image analysis and correlation enhancement. Based on the multi-scale fusion features of the shallow details and deep semantics of the images, the damage level of the mango tree branch phloem is automatically assessed. This approach allows for a more comprehensive and intelligent assessment of the damage to the phloem of mango tree branches, reducing the need for manual assessment and thus improving the efficiency and accuracy of the evaluation.

Description

Technical Field

[0001] The present application relates to the field of intelligent assessment, and more specifically, to a method for assessing phloem damage in mango tree branches. Background Art

[0002] Hainan's off-season mango production uses plant growth regulators such as paclobutrazol and ethephon to control shoots and promote flowering, which can advance the production period by 2-3 months, significantly increase production, and significantly improve economic benefits. However, the problem of abusing plant growth regulators in pursuit of production period and yield is prominent, which can easily lead to tree decline, manifested as gum flow and severe cracking of the phloem of branches and trunks, and even damage to the xylem, resulting in breakage. The phloem of mango branches is an important tissue in the plant body responsible for transporting organic matter downward. If damaged, it will lead to obstruction of nutrient transmission, thereby affecting the nutrient supply and healthy growth of the entire tree, and then affecting the yield and fruit quality. Therefore, repairing damaged mango trees is an important measure to ensure the sustainable development of Hainan's mango industry.

[0003] Before evaluating the damage to the phloem of mango tree branches, it is usually necessary to determine the damage grade to assess the extent and scope of the damage so as to formulate appropriate repair strategies and measures. However, traditional damage grade determination is usually done by manually observing the appearance characteristics of the phloem of mango tree branches, which is prone to subjectivity and cannot accurately assess the depth and scope of damage, resulting in inconsistent and inaccurate determination results, which in turn affects the formulation and implementation of repair strategies.

[0004] Therefore, an optimized method for assessing phloem damage in mango tree branches is desired. Summary of the invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a method for assessing the damage of the phloem of a mango tree trunk, which collects the phloem image of the mango tree trunk by a camera, and uses an image recognition and processing method based on artificial intelligence to perform multi-scale image analysis and mutual correlation enhancement on the phloem image of the mango tree trunk, so as to automatically assess the damage level of the phloem of the mango tree trunk according to the multi-scale fusion features of the shallow details and deep semantics of the phloem image of the mango tree trunk. In this way, the damage to the phloem of the mango tree trunk can be more comprehensively and intelligently assessed, reducing the need for manual assessment, thereby improving the efficiency and accuracy of the assessment.

[0006] According to one aspect of the present application, a method for assessing phloem damage in a mango tree trunk is provided, comprising:

[0007] Assess the extent of damage to the phloem of mango tree branches in order to determine a repair plan, including: obtaining images of the phloem of mango tree branches collected by a camera;

[0008] Performing multi-scale feature extraction on the mango tree trunk phloem image to obtain a mango tree trunk phloem shallow feature map and a mango tree trunk phloem semantic feature map;

[0009] Inputting the mango branch trunk phloem shallow feature map and the mango branch trunk phloem semantic feature map into a collaborative attention module to obtain a collaboratively enhanced mango branch trunk phloem shallow feature map and a collaboratively enhanced mango branch trunk phloem semantic feature map;

[0010] Inputting the synergistically enhanced mango tree trunk phloem shallow feature map and the synergistically enhanced mango tree trunk phloem semantic feature map into a multi-channel feature merging module using semantic information to obtain a mango tree trunk phloem multi-scale fusion feature map as a mango tree trunk phloem multi-scale fusion feature;

[0011] Based on the multi-scale fusion characteristics of the mango tree trunk phloem, an assessment result is obtained, and the assessment result is used to represent a damage grade label.

[0012] Compared with the prior art, the present application provides a method for assessing the damage of the phloem of mango tree branches, which collects phloem images of mango tree branches by a camera, and uses an image recognition and processing method based on artificial intelligence to perform multi-scale image analysis and mutual correlation enhancement on the phloem images of mango tree branches, so as to automatically assess the damage level of the phloem of mango tree branches according to the multi-scale fusion features of the shallow details and deep semantics of the phloem images of mango tree branches. In this way, the damage of the phloem of mango tree branches can be more comprehensively and intelligently assessed, reducing the need for manual assessment, thereby improving the efficiency and accuracy of the assessment. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0014] Figure 1 This is a flow chart for evaluating the degree of damage to the phloem of a mango tree trunk in the method for evaluating damage to the phloem of a mango tree trunk according to an embodiment of the present application to determine a repair plan.

[0015] Figure 2 This is a schematic diagram of a framework for evaluating the degree of damage to the phloem of a mango tree trunk in a method for evaluating damage to the phloem of a mango tree trunk according to an embodiment of the present application to determine a repair solution.

[0016] Figure 3 The present invention provides a flowchart of inputting the mango branch trunk phloem superficial feature map and the mango branch trunk phloem semantic feature map into a collaborative attention module in the mango branch trunk phloem damage assessment method according to an embodiment of the present application to obtain a collaboratively enhanced mango branch trunk phloem superficial feature map and a collaboratively enhanced mango branch trunk phloem semantic feature map.

[0017] Figure 4 The present invention is a flowchart of inputting the synergistically enhanced mango branch trunk phloem shallow feature map and the synergistically enhanced mango branch trunk phloem semantic feature map into a multi-channel feature merging module using semantic information in the mango branch trunk phloem damage assessment method according to an embodiment of the present application to obtain a mango branch trunk phloem multi-scale fusion feature map as a mango branch trunk phloem multi-scale fusion feature.

[0018] Figure 5 The present invention is a flowchart for training the image multi-scale feature extractor based on the hollow pyramid network, the collaborative attention module, the multi-channel feature merging module using semantic information, and the classifier-based damage assessor in the method for assessing phloem damage of mango tree trunks according to an embodiment of the present application. DETAILED DESCRIPTION

[0019] The embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, but rather these embodiments are provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0020] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0021] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0022] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0023] Hainan's off-season mango production uses plant growth regulators such as paclobutrazol and ethephon to control shoots and promote flowering, which can advance the production period by 2-3 months, significantly increase production, and significantly improve economic benefits. However, the problem of abusing plant growth regulators in pursuit of production period and yield is prominent, which can easily lead to tree decline, manifested as gum flow and severe cracking of the phloem of branches and trunks, and even damage to the xylem, resulting in breakage. The phloem of mango branches is an important tissue in the plant body responsible for transporting organic matter downward. If damaged, it will lead to obstruction of nutrient transmission, thereby affecting the nutrient supply and healthy growth of the entire tree, and then affecting the yield and fruit quality. Therefore, repairing damaged mango trees is an important measure to ensure the sustainable development of Hainan's mango industry.

[0024] Before repairing the damaged phloem of mango tree branches, it is usually necessary to determine the damage grade to assess the extent and scope of the damage so as to formulate appropriate repair strategies and measures. However, the traditional damage grade determination is usually done by manually observing the appearance characteristics of the phloem of mango tree branches, which is prone to subjectivity and cannot accurately assess the depth and scope of the damage, resulting in inconsistent and inaccurate determination results, thus affecting the formulation and implementation of repair strategies.

[0025] Based on this, the present application proposes a method for assessing the damage to the phloem of mango tree branches in order to determine a repair plan, which mainly includes: assessing the degree of damage to the phloem of mango tree branches by collecting phloem images of mango tree branches by a camera, and using an artificial intelligence-based image recognition and processing method to perform multi-scale image analysis and mutual correlation enhancement on the phloem images of mango tree branches, so as to automatically assess the damage level of the phloem of mango tree branches based on the multi-scale fusion features of the shallow details and deep semantics of the phloem images of mango tree branches. In this way, the damage to the phloem of mango tree branches can be more comprehensively and intelligently assessed, reducing the need for manual assessment, thereby improving the efficiency and accuracy of the assessment.

[0026] Figure 1 This is a flow chart for evaluating the degree of damage to the phloem of a mango tree trunk in the method for evaluating damage to the phloem of a mango tree trunk according to an embodiment of the present application to determine a repair plan. Figure 2 This is a schematic diagram of the framework for evaluating the degree of damage to the phloem of a mango tree trunk in the method for evaluating damage to the phloem of a mango tree trunk according to an embodiment of the present application to determine a repair solution. Figure 1 and Figure 2As shown, the method for assessing damage to the phloem of a mango branch trunk according to an embodiment of the present application includes: S110, acquiring a phloem image of a mango branch trunk collected by a camera; S120, performing multi-scale feature extraction on the phloem image of the mango branch trunk to obtain a shallow feature map of the phloem of a mango branch trunk and a semantic feature map of the phloem of a mango branch trunk; S130, inputting the shallow feature map of the phloem of a mango branch trunk and the semantic feature map of the phloem of a mango branch trunk into a collaborative attention module to obtain a collaboratively enhanced shallow feature map of the phloem of a mango branch trunk and a collaboratively enhanced semantic feature map of the phloem of a mango branch trunk; S140, inputting the collaboratively enhanced shallow feature map of the phloem of a mango branch trunk and the semantic feature map of the phloem of a mango branch trunk into a multi-channel feature merging module using semantic information to obtain a multi-scale fusion feature map of the phloem of a mango branch trunk as a multi-scale fusion feature of the phloem of a mango branch trunk; and, S150, obtaining an assessment result based on the multi-scale fusion feature of the phloem of a mango branch trunk, wherein the assessment result is used to represent a damage level label.

[0027] In step S110, a phloem image of a mango tree trunk captured by a camera is obtained. It should be understood that the phloem of a mango tree trunk is an important tissue in the plant stem, responsible for transporting organic substances produced in the plant body, such as glucose, amino acids, etc., which are necessary for plant growth and development. Based on this, in order to observe the damage to the phloem of a mango tree trunk, such as cuts, cracks, diseases and insect pests, etc., to facilitate a more accurate damage assessment, in the technical solution of the present application, a phloem image of a mango tree trunk captured by a camera is obtained.

[0028] In step S120, multi-scale feature extraction is performed on the mango branch trunk phloem image to obtain a shallow feature map of the mango branch trunk phloem and a semantic feature map of the mango branch trunk phloem. Specifically, in an embodiment of the present application, multi-scale feature extraction is performed on the mango branch trunk phloem image to obtain a shallow feature map of the mango branch trunk phloem and a semantic feature map of the mango branch trunk phloem, including: passing the mango branch trunk phloem image through an image multi-scale feature extractor based on a hollow pyramid network to obtain the shallow feature map of the mango branch trunk phloem and the semantic feature map of the mango branch trunk phloem. Accordingly, considering that the mango branch trunk phloem image contains feature information about shallow details and deep semantics of phloem at different scales, such as shallow detail feature information on the roughness and color changes of the bark; different types of damage, such as deep semantic information on cuts, cracks, signs of diseases and insect pests, etc. The hollow pyramid network has good image feature extraction capabilities at different scales, including local details and global semantic information, which helps to fully understand the structure and characteristics of the mango tree trunk phloem image, so as to more accurately evaluate the damage level. Therefore, in the technical solution of the present application, the mango tree trunk phloem image is passed through an image multi-scale feature extractor based on the hollow pyramid network to effectively capture and mine the multi-scale features of the mango tree trunk phloem image, thereby obtaining a mango tree trunk phloem shallow feature map and a mango tree trunk phloem semantic feature map, providing a more accurate and comprehensive information basis for the automatic evaluation of the damage level.

[0029] In step S130, the mango branch phloem shallow feature map and the mango branch phloem semantic feature map are input into the collaborative attention module to obtain the collaboratively enhanced mango branch phloem shallow feature map and the collaboratively enhanced mango branch phloem semantic feature map. It should be understood that, considering that there are some similar information and associations between the mango branch phloem shallow feature map and the mango branch phloem semantic feature map, such as some mango branch phloem local detail features may be associated with the overall semantic information. Based on this, in order to enhance the features of the mango branch phloem shallow feature map and the mango branch phloem semantic feature map based on the similarity information between the two, in the technical solution of the present application, the mango branch phloem shallow feature map and the mango branch phloem semantic feature map are input into the collaborative attention module to obtain the collaboratively enhanced mango branch phloem shallow feature map and the collaboratively enhanced mango branch phloem semantic feature map. In detail, the collaborative attention module first calculates the similarity information of the shallow feature map of the mango branch phloem and the semantic feature map of the mango branch phloem on the whole channel to quantify the similarity and correlation between the two feature maps, and obtains a similarity matrix. Then, the shallow feature map of the mango branch phloem and the semantic feature map of the mango branch phloem are respectively subjected to global mean pooling to capture the statistical information and global features of the overall features, and obtain a shallow full-channel feature vector and a semantic full-channel feature vector. Then, the normalized similarity matrix is ​​matrix-multiplied with the shallow full-channel feature vector and the semantic full-channel feature vector to combine the correlation information between different feature maps and perform information interaction, and obtain the corresponding shallow and semantic mutual collaborative attention features. Finally, the obtained shallow and semantic mutual collaborative attention features are used as weights to weight the shallow feature map of the mango branch phloem and the semantic feature map of the mango branch phloem to achieve collaborative reinforcement between the shallow feature map and the semantic feature map of the mango branch phloem, so as to further improve the accuracy and comprehensiveness of the assessment of the damage level.

[0030] Figure 3 The present invention is a flowchart of inputting the mango branch trunk phloem superficial feature map and the mango branch trunk phloem semantic feature map into the collaborative attention module in the mango branch trunk phloem damage assessment method according to the embodiment of the present application to obtain the collaboratively enhanced mango branch trunk phloem superficial feature map and the collaboratively enhanced mango branch trunk phloem semantic feature map. Specifically, in the embodiment of the present application, Figure 3As shown, the mango branch trunk phloem shallow feature map and the mango branch trunk phloem semantic feature map are input into the collaborative attention module to obtain the collaboratively enhanced mango branch trunk phloem shallow feature map and the collaboratively enhanced mango branch trunk phloem semantic feature map, including: S210, reshaping the mango branch trunk phloem shallow feature map and the mango branch trunk phloem semantic feature map to obtain the mango branch trunk phloem shallow feature matrix and the mango branch trunk phloem semantic feature matrix; S220, calculating the product between the transposed matrix of the mango branch trunk phloem shallow feature matrix and the mango branch trunk phloem semantic feature matrix to obtain the full-channel mango branch trunk phloem similarity between the mango branch trunk phloem shallow feature map and the mango branch trunk phloem semantic feature map matrix; S230, normalizing the full-channel mango branch phloem similarity matrix to obtain a normalized full-channel mango branch phloem similarity matrix; S240, calculating a mango branch phloem shallow collaborative attention weight vector and a mango branch phloem semantic collaborative attention weight vector based on the normalized full-channel mango branch phloem similarity matrix; and, S250, using the mango branch phloem shallow collaborative attention weight vector and the mango branch phloem semantic collaborative attention weight vector as weight vectors, respectively calculating the channel-by-channel product of the mango branch phloem shallow feature map and the mango branch phloem semantic feature map to obtain the collaboratively enhanced mango branch phloem shallow feature map and the collaboratively enhanced mango branch phloem semantic feature map.

[0031] More specifically, in an embodiment of the present application, the full-channel mango branch phloem similarity matrix is ​​normalized to obtain a normalized full-channel mango branch phloem similarity matrix, including: taking each eigenvalue in the full-channel mango branch phloem similarity matrix as an exponent of a natural constant to calculate a position-based exponential function value with the natural constant as the base to obtain a mango branch phloem full-channel feature similarity class support matrix; calculating the sum of each column vector in the mango branch phloem full-channel feature similarity class support matrix to obtain a mango branch phloem full-channel feature similarity class support column global vector composed of multiple column vector global sum values; and calculating the position-based division between each eigenvalue in the mango branch phloem full-channel feature similarity class support matrix and the corresponding column vector global sum value in the mango branch phloem full-channel feature similarity class support column global vector to obtain the normalized full-channel mango branch phloem similarity matrix.

[0032] More specifically, in an embodiment of the present application, a mango branch trunk phloem shallow collaborative attention weight vector and a mango branch trunk phloem semantic collaborative attention weight vector are calculated based on the normalized full-channel mango branch trunk phloem similarity matrix, including: performing global mean pooling along the channel dimension on the mango branch trunk phloem shallow feature map and the mango branch trunk phloem semantic feature map to obtain a mango branch trunk phloem shallow full-channel feature vector and a mango branch trunk phloem semantic full-channel feature vector; calculating the matrix product between the normalized full-channel mango branch trunk phloem similarity matrix and the mango branch trunk phloem shallow full-channel feature vector to obtain the mango branch trunk phloem shallow collaborative attention weight vector; and, calculating the matrix product between the normalized full-channel mango branch trunk phloem similarity matrix and the mango branch trunk phloem semantic full-channel feature vector to obtain the mango branch trunk phloem semantic collaborative attention weight vector.

[0033] In an embodiment of the present application, specifically, the mango branch trunk phloem shallow feature map and the mango branch trunk phloem semantic feature map are input into a collaborative attention module to obtain a collaboratively enhanced mango branch trunk phloem shallow feature map and a collaboratively enhanced mango branch trunk phloem semantic feature map, including: using the collaborative attention module to process the mango branch trunk phloem shallow feature map and the mango branch trunk phloem semantic feature map with the following collaborative attention formula to obtain the collaboratively enhanced mango branch trunk phloem shallow feature map and the collaboratively enhanced mango branch trunk phloem semantic feature map; wherein the collaborative attention formula is:

[0034] F1′=Reshape(F1)

[0035] F2′=Reshape(F2)

[0036] M=F1′×F2′ T

[0037]

[0038] C1=Avgpooling(F1)

[0039] C2=Avgpooling(F2)

[0040] a1=W×C1

[0041] a2=W×C2

[0042]

[0043] Among them, F1 and F2 are the shallow feature map of the mango branch phloem and the semantic feature map of the mango branch phloem, respectively; Reshape is the shape reshaping process; F1′ and F2′ are the shallow feature matrix of the mango branch phloem and the semantic feature matrix of the mango branch phloem, respectively; M is the full-channel mango branch phloem similarity matrix, M ij and M ik are the eigenvalues ​​of each position in the full-channel mango tree trunk phloem similarity matrix, exp(·) is an exponential operation with the natural constant e as the base, c is the number of row vectors in the full-channel mango tree trunk phloem similarity matrix, W i,j is the eigenvalue of each position in the normalized full-channel mango branch phloem similarity matrix, Avgpooling is the global mean pooling process of the feature map, C1 and C2 are the shallow full-channel feature vector of the mango branch phloem and the semantic full-channel feature vector of the mango branch phloem, W is the normalized full-channel mango branch phloem similarity matrix, a1 and a2 are the shallow collaborative attention weight vector of the mango branch phloem and the semantic collaborative attention weight vector of the mango branch phloem, F a1 and F a2 They are the synergistically enhanced mango tree trunk phloem shallow feature map and the synergistically enhanced mango tree trunk phloem semantic feature map, Represents matrix multiplication.

[0044] In step S140, the synergistically enhanced mango branch phloem shallow feature map and the synergistically enhanced mango branch phloem semantic feature map are input into a multi-channel feature merging module using semantic information to obtain a multi-scale fusion feature map of mango branch phloem as a multi-scale fusion feature of mango branch phloem. Accordingly, considering that the mango branch phloem shallow feature map captures the low-level information and local details of the mango branch phloem image, it reflects the specific details and structural features of the mango branch phloem, but lacks the grasp of the overall semantic information. The mango branch phloem semantic feature map focuses more on the high-level semantic information and overall structure of the mango branch phloem image, which can reflect the overall meaning and semantic content of the mango branch phloem, but lacks the understanding of local fine-grainedness. Therefore, in order to be able to better fuse and integrate the features of the two based on the respective advantage information between the two features, in the technical solution of the present application, the synergistically enhanced mango branch phloem shallow feature map and the synergistically enhanced mango branch phloem semantic feature map are input into a multi-channel feature merging module using semantic information to obtain a multi-scale fusion feature map of mango branch phloem. Specifically, the multi-channel feature merging module using semantic information first performs convolution coding and normalization processing on the feature map after cascading the synergistically enhanced mango branch phloem shallow feature map and the synergistically enhanced mango branch phloem semantic feature map, so as to effectively integrate the mango branch phloem feature information of different scales and levels to capture richer and more abstract feature information, and obtain normalized associated features. Then, the normalized associated features are subjected to multiple pooling and weighted fusion to highlight the key information of different levels and scales in the mango branch phloem associated features, so that the mango branch phloem features are more diverse and rich to obtain multi-scale pooling features. Finally, the multi-scale pooling features are used to perform feature weighting on the shallow feature map of the mango tree trunk phloem to improve the expression ability and accuracy of the fine-grained structure of the mango tree trunk phloem, thereby providing a more reliable basis for damage assessment and repair.

[0045] Figure 4 The present invention is a flowchart of inputting the synergistically enhanced mango branch trunk phloem shallow feature map and the synergistically enhanced mango branch trunk phloem semantic feature map into a multi-channel feature merging module using semantic information to obtain a mango branch trunk phloem multi-scale fusion feature map as a mango branch trunk phloem multi-scale fusion feature in the mango branch trunk phloem damage assessment method according to the embodiment of the present application. Specifically, in the embodiment of the present application, Figure 4As shown, the synergistically enhanced mango branch phloem shallow feature map and the synergistically enhanced mango branch phloem semantic feature map are input into a multi-channel feature merging module using semantic information to obtain a mango branch phloem multi-scale fusion feature map as a mango branch phloem multi-scale fusion feature, including: S310, cascading the synergistically enhanced mango branch phloem shallow feature map and the synergistically enhanced mango branch phloem semantic feature map to obtain a synergistically enhanced mango branch phloem shallow semantic series feature map; S320, convolutionally encoding the synergistically enhanced mango branch phloem shallow semantic series feature map to obtain a mango branch phloem shallow semantic association feature map; S330, batch normalizing the mango branch phloem shallow semantic association feature map to obtain a normalized mango branch phloem shallow semantic association feature map; S340, performing maximum-based global pooling on the normalized mango branch phloem shallow semantic association feature map S350, calculating the weighted sum by position between the shallow semantic global maximum pooling feature vector of mango branch phloem, the shallow semantic global random value pooling feature vector of mango branch phloem and the shallow semantic global mean pooling feature vector of mango branch phloem; S350, calculating the weighted sum by position between the shallow semantic global maximum pooling feature vector of mango branch phloem, the shallow semantic global random value pooling feature vector of mango branch phloem and the shallow semantic global mean pooling feature vector of mango branch phloem to obtain the shallow semantic association multi-scale pooling representation vector of mango branch phloem; and, S360, using the feature value of each position in the shallow semantic association multi-scale pooling representation vector of mango branch phloem as the weight, weighting each feature matrix along the channel dimension of the synergistically enhanced shallow feature map of mango branch phloem is performed to obtain the multi-scale fusion feature map of mango branch phloem.

[0046] More specifically, in an embodiment of the present application, the position-weighted sum of the shallow semantic global maximum pooling feature vector of the mango branch trunk phloem, the shallow semantic global random value pooling feature vector of the mango branch trunk phloem and the shallow semantic global mean pooling feature vector of the mango branch trunk phloem is calculated to obtain a shallow semantic associated multi-scale pooling representation vector of the mango branch trunk phloem, including: the shallow semantic global maximum pooling feature vector of the mango branch trunk phloem, the shallow semantic global random value pooling feature vector of the mango branch trunk phloem and the shallow semantic global mean pooling feature vector of the mango branch trunk phloem are pooled. The quantities are respectively multiplied by the corresponding adjustment parameters by position to obtain the global maximum modulation feature vector of the shallow semantics of the mango branch phloem, the global random value modulation feature vector of the shallow semantics of the mango branch phloem and the global mean modulation feature vector of the shallow semantics of the mango branch phloem; and the global maximum modulation feature vector of the shallow semantics of the mango branch phloem, the global random value modulation feature vector of the shallow semantics of the mango branch phloem and the global mean modulation feature vector of the shallow semantics of the mango branch phloem are added by position to obtain the multi-scale pooling representation vector of the shallow semantic association of the mango branch phloem.

[0047] In an embodiment of the present application, specifically, the synergistically enhanced mango branch trunk phloem shallow feature map and the synergistically enhanced mango branch trunk phloem semantic feature map are input into a multi-channel feature merging module using semantic information to obtain a multi-scale fusion feature map of mango branch trunk phloem, including: using the multi-channel feature merging module using semantic information to process the synergistically enhanced mango branch trunk phloem shallow feature map and the synergistically enhanced mango branch trunk phloem semantic feature map using the following multi-channel feature merging formula to obtain the multi-scale fusion feature map of mango branch trunk phloem; wherein the multi-channel feature merging formula is:

[0048] F s =BN(Conv(concat(F a1 ,F a2 )))

[0049] S v =α·AvgPool(F s )+β·MaxPool(F s )+γ·StoPool(F s )

[0050] F av =attention(S v ,F a1 )

[0051] Among them, F a1 and F a2are the synergistically enhanced mango tree trunk phloem shallow feature map and the synergistically enhanced mango tree trunk phloem semantic feature map, concat(·) is the cascade processing of the feature map, Conv(·) is the convolution coding of the feature map, BN(·) represents batch normalization processing, F s is the normalized mango tree trunk phloem shallow semantic association feature map, AvgPool(·), MaxPool(·) and StoPool(·) are global pooling processing based on average value, global pooling processing based on maximum value and global pooling processing based on random value for each feature matrix along the channel dimension of the feature map, α, β and γ are the corresponding adjustment parameters, S v is the multi-scale pooling representation vector of the shallow semantic association of the mango tree trunk phloem, F av is the multi-scale fusion feature map of the mango tree trunk phloem, and attention(·) represents the weighted operation of the feature map using a vector.

[0052] In step S150, based on the multi-scale fusion features of the mango tree trunk phloem, an evaluation result is obtained, and the evaluation result is used to represent the damage level label. Specifically, in the embodiment of the present application, based on the multi-scale fusion features of the mango tree trunk phloem, an evaluation result is obtained, including: inputting the multi-scale fusion feature map of the mango tree trunk phloem into a damage assessor based on a classifier to obtain the evaluation result. That is, the multi-scale fusion features of the mango tree trunk phloem obtained by multi-channel feature fusion of the synergistically enhanced mango tree trunk phloem shallow feature map and the synergistically enhanced mango tree trunk phloem semantic feature map are used for classification processing to automatically evaluate the damage level of the mango tree trunk phloem. In this way, the damage to the mango tree trunk phloem can be more comprehensively and intelligently evaluated, reducing the need for manual evaluation, thereby improving the efficiency and accuracy of the evaluation.

[0053] It is worth mentioning that those skilled in the art should be aware that before applying a deep neural network model for inference, the deep neural network model must be trained so that the deep neural network can implement specific functional capabilities.

[0054] Specifically, in an embodiment of the present application, a training step is also included: for training the image multi-scale feature extractor based on the hollow pyramid network, the collaborative attention module, the multi-channel feature merging module using semantic information, and the classifier-based damage assessor.

[0055] Figure 5The present invention is a flowchart of training the multi-scale image feature extractor based on the hollow pyramid network, the collaborative attention module, the multi-channel feature merging module using semantic information, and the damage assessor based on the classifier in the method for assessing the damage to the phloem of a mango tree trunk according to an embodiment of the present application. Figure 5 As shown, the training step includes: S410, obtaining training data, wherein the training data includes a training mango tree trunk phloem image collected by a camera, and a true evaluation result, wherein the true evaluation result is a true grade label of damage; S420, passing the training mango tree trunk phloem image through the image multi-scale feature extractor based on the hollow pyramid network to obtain a training mango tree trunk phloem shallow feature map and a training mango tree trunk phloem semantic feature map; S430, inputting the training mango tree trunk phloem shallow feature map and the training mango tree trunk phloem semantic feature map into the collaborative attention module to obtain a training collaboratively enhanced mango tree trunk phloem shallow feature map and a training collaboratively enhanced mango tree trunk phloem semantic feature map; S440, inputting the training collaboratively enhanced mango tree trunk phloem shallow feature map and the training collaboratively enhanced mango tree trunk phloem semantic feature map into the collaborative attention module to obtain a training collaboratively enhanced mango tree trunk phloem shallow feature map and a training collaboratively enhanced mango tree trunk phloem semantic feature map; The shallow feature map of branch phloem and the trained collaboratively enhanced semantic feature map of mango branch trunk phloem are input into the multi-channel feature merging module using semantic information to obtain the multi-scale fusion feature map of the training mango branch trunk phloem; S450, the multi-scale fusion feature map of the training mango branch trunk phloem is passed through the classifier-based damage assessor to obtain a training evaluation result; S460, the cross entropy loss function value between the training evaluation result and the true evaluation result is calculated to obtain a classification loss function value; and, S470, based on the classification loss function value and through back propagation of gradient descent, the image multi-scale feature extractor based on the hollow pyramid network, the collaborative attention module, the multi-channel feature merging module using semantic information and the classifier-based damage assessor are trained.

[0056] It should be understood that in the embodiment of the present application, for the cross-scale and cross-depth image semantic feature distribution differences represented by the collaborative attention of the cross-scale and cross-depth image semantic features of the mango branch phloem image respectively represented by the trained collaboratively enhanced mango trunk phloem shallow feature map and the trained collaboratively enhanced mango branch phloem semantic feature map, after the multi-channel feature merging, the training mango branch phloem multi-scale fusion feature map will also have the problem of insufficient distribution aggregation caused by the differentiated image semantic feature channel change distribution, thereby affecting its classification convergence efficiency through the classifier-based damage assessor, that is, affecting the efficiency of classification training and the accuracy of classification results.

[0057] Therefore, preferably, the multi-scale fusion feature map of the phloem of the training mango branch is input into the damage assessor based on the classifier to obtain the training assessment result, which specifically includes the following optimization steps: multiplying the multi-scale fusion feature map of the phloem of the training mango branch by the scale of the multi-scale fusion feature map of the phloem of the training mango branch, and then performing point subtraction with the sum of the absolute values ​​of each feature value of the multi-scale fusion feature map of the phloem of the training mango branch to obtain the multi-scale fusion full semantic feature map of the phloem of the training mango branch, wherein the multi-scale fusion feature map of the phloem of the training mango branch is The scale of the fusion feature map is the width of each feature matrix of the training mango branch phloem multi-scale fusion feature map along the channel dimension multiplied by the height and then multiplied by the number of channels of the training mango branch phloem multi-scale fusion feature map; the training mango branch phloem multi-scale fusion feature map is multiplied by the square root of the scale of the training mango branch phloem multi-scale fusion feature map, and then the square root of the sum of the squares of the eigenvalues ​​of the training mango branch phloem multi-scale fusion feature map is subtracted from the square root of the sum of the squares of the eigenvalues ​​of the training mango branch phloem multi-scale fusion feature map to obtain the training mango branch phloem multi-scale fusion semi-semantic feature map; The square root of the absolute value of each eigenvalue of the multi-scale fusion full semantic feature map of the training mango branch phloem and the multi-scale fusion semi-semantic feature map of the training mango branch phloem is calculated respectively to obtain the multi-scale fusion full semantic modulation feature map of the training mango branch phloem and the multi-scale fusion semi-semantic modulation feature map of the training mango branch phloem; the logarithm with base 2 of each eigenvalue of the multi-scale fusion full semantic modulation feature map of the training mango branch phloem and the multi-scale fusion semi-semantic modulation feature map of the training mango branch phloem is calculated respectively to obtain the multi-scale fusion full semantic modulation feature map of the training mango branch phloem and the multi-scale fusion semi-semantic modulation feature map of the training mango branch phloem A multi-scale fusion full semantic information feature map of the phloem of a tree branch and a multi-scale fusion semi-semantic information feature map of the phloem of a training mango tree branch; a weighted sum of the multi-scale fusion full semantic information feature map of the phloem of a training mango tree branch and a multi-scale fusion semi-semantic information feature map of the phloem of a training mango tree branch is calculated using a weighting coefficient as a hyperparameter to obtain the optimized multi-scale fusion feature map of the phloem of a training mango tree branch; and, the optimized multi-scale fusion feature map of the phloem of a training mango tree branch and a damage assessor based on the classifier is input to obtain the training assessment result.

[0058] That is, the multi-scale fusion feature map of the phloem of the training mango branch trunk is used as a feature set, in which the change semantic representation is based on the feature value of the multi-scale fusion feature map of the phloem of the training mango branch trunk. In order to dynamically aggregate the semantic set composed of different change semantics of the multi-scale fusion feature map of the phloem of the training mango branch trunk without ignoring the individual semantic change information, the individual features of the multi-scale fusion feature map of the phloem of the training mango branch trunk and the collective expression of its feature map scale are used as full-width and half-width feature representations, and the low-rank norms of different dimensions of the overall semantics of the feature set of the multi-scale fusion feature map of the phloem of the training mango branch trunk are used as phases through negative correlation calculation and amplitude-phase information representation modulation is performed to dynamically adjust the semantic information change relationship of the multi-scale fusion feature map of the phloem of the training mango branch trunk, thereby improving the overall semantic information expression aggregation of the multi-scale fusion feature map of the phloem of the training mango branch trunk, and improving the classification convergence efficiency of the multi-scale fusion feature map of the phloem of the training mango branch trunk when it is classified by the classifier-based damage assessor. In this way, the damage to the phloem of mango tree trunks can be assessed more comprehensively and intelligently, reducing the need for manual assessment and thus improving the efficiency and accuracy of the assessment.

[0059] In an embodiment of the present application, specifically, when the multi-scale fusion feature map of the training mango branch trunk phloem is input into a damage assessor based on a classifier to obtain a training assessment result, the multi-scale fusion feature map of the training mango branch trunk phloem is optimized to obtain the optimized multi-scale fusion feature map of the training mango branch trunk phloem, including: optimizing the multi-scale fusion feature map of the training mango branch trunk phloem using the following optimization formula to obtain the optimized multi-scale fusion feature map of the training mango branch trunk phloem; wherein the optimization formula is:

[0060]

[0061] Among them, F is the multi-scale fusion feature map of the training mango branch phloem, L is the scale of the multi-scale fusion feature map of the training mango branch phloem, that is, the width of each feature matrix along the channel dimension of the training mango branch phloem multi-scale fusion feature map ... is the point-by-point subtraction, ‖·‖2 is the bi-norm of the feature map, ω is the hyperparameter, It is added by position, log represents the logarithmic function value with base 2, |·| ⊙1 / 2 represents the square root of the absolute value of the feature value at each position in the calculated feature map, and F′ is the optimized multi-scale fusion feature map of the phloem of the training mango tree trunk.

[0062] In summary, the method for assessing the damage of the phloem of the mango tree trunk based on the embodiment of the present application is explained, which collects the phloem image of the mango tree trunk by a camera, and uses an image recognition and processing method based on artificial intelligence to perform multi-scale image analysis and mutual correlation enhancement on the phloem image of the mango tree trunk, so as to automatically assess the damage level of the phloem of the mango tree trunk according to the multi-scale fusion features of the shallow details and deep semantics of the phloem image of the mango tree trunk. In this way, the damage of the phloem of the mango tree trunk can be more comprehensively and intelligently assessed, reducing the need for manual assessment, thereby improving the efficiency and accuracy of the assessment.

[0063] The foregoing is merely an example of the principles of the present disclosure, and various modifications may be made by those skilled in the art without departing from the scope of the present disclosure. The above embodiments are presented for the purpose of illustration and not limitation. The present disclosure may also take many forms other than those explicitly described herein. Therefore, it is emphasized that the present disclosure is not limited to the methods, systems and devices explicitly disclosed, but is intended to include variations and modifications within the spirit and scope of the appended claims.

Claims

1. A method for assessing damage to the phloem of a mango tree trunk, which assesses the degree of damage to the phloem of a mango tree trunk in order to determine a repair plan; characterized in that: include: Obtain the phloem image of the mango tree trunk collected by the camera; Performing multi-scale feature extraction on the mango tree trunk phloem image to obtain a mango tree trunk phloem shallow feature map and a mango tree trunk phloem semantic feature map; Inputting the mango tree trunk phloem superficial feature map and the mango tree trunk phloem semantic feature map into a collaborative attention module to obtain a collaboratively enhanced mango tree trunk phloem superficial feature map and a collaboratively enhanced mango tree trunk phloem semantic feature map; Inputting the synergistically enhanced mango tree trunk phloem shallow feature map and the synergistically enhanced mango tree trunk phloem semantic feature map into a multi-channel feature merging module using semantic information to obtain a mango tree trunk phloem multi-scale fusion feature map as a mango tree trunk phloem multi-scale fusion feature; Based on the multi-scale fusion characteristics of the mango tree trunk phloem, an assessment result is obtained, and the assessment result is used to represent a damage grade label; Performing multi-scale feature extraction on the mango branch trunk phloem image to obtain a shallow feature map of the mango branch trunk phloem and a semantic feature map of the mango branch trunk phloem, including: passing the mango branch trunk phloem image through an image multi-scale feature extractor based on a hollow pyramid network to obtain the shallow feature map of the mango branch trunk phloem and the semantic feature map of the mango branch trunk phloem; Inputting the mango tree trunk phloem superficial feature map and the mango tree trunk phloem semantic feature map into a collaborative attention module to obtain a collaboratively enhanced mango tree trunk phloem superficial feature map and a collaboratively enhanced mango tree trunk phloem semantic feature map, including: Reshape the mango branch trunk phloem shallow feature map and the mango branch trunk phloem semantic feature map to obtain a mango branch trunk phloem shallow feature matrix and a mango branch trunk phloem semantic feature matrix; Calculate the product between the transposed matrix of the mango branch trunk phloem shallow feature matrix and the mango branch trunk phloem semantic feature matrix to obtain a full-channel mango branch trunk phloem similarity matrix between the mango branch trunk phloem shallow feature map and the mango branch trunk phloem semantic feature map; Normalizing the full-channel mango tree trunk phloem similarity matrix to obtain a normalized full-channel mango tree trunk phloem similarity matrix; Calculating a mango tree trunk phloem shallow collaborative attention weight vector and a mango tree trunk phloem semantic collaborative attention weight vector based on the normalized full-channel mango tree trunk phloem similarity matrix; Taking the mango branch trunk phloem shallow collaborative attention weight vector and the mango branch trunk phloem semantic collaborative attention weight vector as weight vectors, respectively calculating the channel-by-channel product of the mango branch trunk phloem shallow feature map and the mango branch trunk phloem semantic feature map to obtain the collaboratively enhanced mango branch trunk phloem shallow feature map and the collaboratively enhanced mango branch trunk phloem semantic feature map; The full-channel mango tree trunk phloem similarity matrix is ​​normalized to obtain a normalized full-channel mango tree trunk phloem similarity matrix, comprising: Taking each eigenvalue in the full-channel mango tree trunk phloem similarity matrix as an exponent of a natural constant to calculate an exponential function value based on the natural constant according to position to obtain a mango tree trunk phloem full-channel feature similarity class support matrix; Calculate the sum of each column vector in the mango branch trunk phloem full channel feature similarity class support matrix to obtain a mango branch trunk phloem full channel feature similarity class support column global vector composed of multiple column vector global sum values; Calculate the positional division between each eigenvalue in the mango branch phloem full channel feature similarity class support matrix and the corresponding column vector global sum value in the mango branch phloem full channel feature similarity class support column global vector to obtain the normalized mango branch phloem full channel similarity matrix; Calculating the mango tree trunk phloem shallow collaborative attention weight vector and the mango tree trunk phloem semantic collaborative attention weight vector based on the normalized full-channel mango tree trunk phloem similarity matrix includes: Performing global mean pooling along the channel dimension on the mango tree trunk phloem shallow feature map and the mango tree trunk phloem semantic feature map to obtain a mango tree trunk phloem shallow full channel feature vector and a mango tree trunk phloem semantic full channel feature vector; Calculating the matrix product between the normalized full-channel mango branch trunk phloem similarity matrix and the shallow full-channel feature vector of the mango branch trunk phloem to obtain the mango branch trunk phloem shallow collaborative attention weight vector; Calculating the matrix product between the normalized full-channel mango branch stem phloem similarity matrix and the mango branch stem phloem semantic full-channel feature vector to obtain the mango branch stem phloem semantic collaborative attention weight vector; The synergistically enhanced mango tree trunk phloem shallow feature map and the synergistically enhanced mango tree trunk phloem semantic feature map are input into a multi-channel feature merging module using semantic information to obtain a mango tree trunk phloem multi-scale fusion feature map as a mango tree trunk phloem multi-scale fusion feature, including: Cascading the synergistically enhanced mango tree trunk phloem shallow feature map and the synergistically enhanced mango tree trunk phloem semantic feature map to obtain a synergistically enhanced mango tree trunk phloem shallow semantic series feature map; Convolutionally encoding the synergistically enhanced mango tree trunk phloem shallow semantic series feature map to obtain a mango tree trunk phloem shallow semantic association feature map; performing batch normalization processing on the shallow semantic association feature map of the mango tree trunk phloem to obtain a normalized shallow semantic association feature map of the mango tree trunk phloem; The normalized mango tree trunk phloem shallow semantic association feature map is subjected to maximum-based global pooling processing, random-value-based global pooling processing, and average-based global pooling processing to obtain a mango tree trunk phloem shallow semantic global maximum pooling feature vector, a mango tree trunk phloem shallow semantic global random-value pooling feature vector, and a mango tree trunk phloem shallow semantic global average pooling feature vector; Calculate the position-weighted sum of the shallow semantic global maximum pooling feature vector of the mango branch trunk phloem, the shallow semantic global random value pooling feature vector of the mango branch trunk phloem, and the shallow semantic global mean pooling feature vector of the mango branch trunk phloem to obtain a mango branch trunk phloem shallow semantic association multi-scale pooling representation vector; Taking the eigenvalues ​​of each position in the shallow semantic association multi-scale pooling representation vector of the mango branch trunk phloem as weights, each feature matrix along the channel dimension of the synergistically enhanced mango branch trunk phloem shallow feature map is weighted to obtain the multi-scale fusion feature map of the mango branch trunk phloem.

2. The method for assessing phloem damage in mango tree branches according to claim 1, characterized in that: Calculating the position-weighted sum of the shallow semantic global maximum pooling feature vector of the mango branch trunk phloem, the shallow semantic global random value pooling feature vector of the mango branch trunk phloem, and the shallow semantic global mean pooling feature vector of the mango branch trunk phloem to obtain a mango branch trunk phloem shallow semantic association multi-scale pooling representation vector, including: The shallow semantic global maximum pooling feature vector of the mango branch phloem, the shallow semantic global random value pooling feature vector of the mango branch phloem and the shallow semantic global mean pooling feature vector of the mango branch phloem are respectively multiplied by the corresponding adjustment parameters by position to obtain the shallow semantic global maximum modulation feature vector of the mango branch phloem, the shallow semantic global random value modulation feature vector of the mango branch phloem and the shallow semantic global mean modulation feature vector of the mango branch phloem; The shallow semantic global maximum modulation feature vector of the mango branch trunk phloem, the shallow semantic global random value modulation feature vector of the mango branch trunk phloem and the shallow semantic global mean modulation feature vector of the mango branch trunk phloem are added by position to obtain the shallow semantic association multi-scale pooling representation vector of the mango branch trunk phloem.

3. The method for assessing phloem damage in mango tree branches according to claim 2, characterized in that: Based on the multi-scale fusion features of the mango tree trunk phloem, an assessment result is obtained, including: inputting the multi-scale fusion feature map of the mango tree trunk phloem into a damage assessor based on a classifier to obtain the assessment result.

4. The method for assessing phloem damage in mango tree branches according to claim 3, characterized in that: The method also includes a training step for training the image multi-scale feature extractor based on the dilated pyramid network, the collaborative attention module, the multi-channel feature merging module using semantic information, and the classifier-based damage assessor.

5. The method for assessing phloem damage in mango tree branches according to claim 4, characterized in that: The training step comprises: Acquire training data, wherein the training data includes a training mango tree trunk phloem image captured by a camera, and a true assessment result, wherein the true assessment result is a true grade label of damage; The training mango tree trunk phloem image is passed through the image multi-scale feature extractor based on the hollow pyramid network to obtain a training mango tree trunk phloem shallow feature map and a training mango tree trunk phloem semantic feature map; Inputting the training mango branch trunk phloem shallow feature map and the training mango branch trunk phloem semantic feature map into the collaborative attention module to obtain a training collaboratively enhanced mango branch trunk phloem shallow feature map and a training collaboratively enhanced mango branch trunk phloem semantic feature map; Inputting the training collaboratively enhanced mango tree trunk phloem shallow feature map and the training collaboratively enhanced mango tree trunk phloem semantic feature map into the multi-channel feature merging module using semantic information to obtain a training mango tree trunk phloem multi-scale fusion feature map; The multi-scale fusion feature map of the phloem of the training mango tree trunk is passed through the classifier-based damage assessor to obtain a training assessment result; Calculating a cross entropy loss function value between the training evaluation result and the true evaluation result to obtain a classification loss function value; The image multi-scale feature extractor based on the dilated pyramid network, the collaborative attention module, the multi-channel feature merging module using semantic information, and the classifier-based damage assessor are trained based on the classification loss function value and through back propagation of gradient descent.

Images