Plant leaf image recognition method based on contour shape and depth feature fusion
By fusing contour shape and depth features, multi-scale multi-contour distance features are extracted and combined with high-level semantic features, solving the problems of large sample size, long training time and poor recognition effect in existing leaf image recognition technologies, and realizing efficient leaf image recognition and retrieval.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- JIANGSU OPEN UNIVERSITY (THE CITY VOCATIONAL COLLEGE OF JIANGSU)
- Filing Date
- 2022-11-23
- Publication Date
- 2026-06-30
AI Technical Summary
Existing technologies for plant leaf image recognition suffer from problems such as large sample size, long training time, unclear mechanism, and poor recognition effect with small samples. Furthermore, local shape descriptors have difficulty recognizing leaf shapes with small inter-class differences and large intra-class variations.
A method based on contour shape and depth feature fusion is adopted. By extracting minimum projection distance, relative projection distance and arch height distance features, and combining multi-scale analysis, low-level contour features and high-level semantic features are fused. A pre-trained VGG16 model is used to extract high-level semantic features, and a maximum value normalization strategy is used to fuse features at different levels.
It significantly improves the performance of blade image recognition, achieving invariance to translation, scaling, and rotation of blade images, thereby enhancing recognition accuracy and retrieval performance.
Smart Images
Figure CN115953615B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of plant leaf recognition technology, and more specifically, relates to a plant leaf image recognition method based on the fusion of contour shape and depth features. Background Technology
[0002] Analyzing leaf shape patterns and extracting recognizable leaf shape descriptors has become an important direction in leaf image pattern recognition in recent years. Existing leaf shape descriptors can be divided into two categories: global shape descriptors and local shape descriptors. Global shape descriptors extract features such as geometric invariant moments and Zernike moments of the entire region's shape, but suffer from drawbacks such as large data volume and high computational complexity. Local shape descriptors, on the other hand, extract the spatial distribution and geometric features of the contour shape, offering advantages such as compact descriptors and low computational complexity. These descriptors have attracted considerable attention from researchers and have become a focus of image retrieval research.
[0003] Local feature description methods mainly include point-set-based methods and curve-based methods. Point-set-based methods treat the contour boundary as an unordered set of points and extract the relative spatial distribution features of the boundary points to construct a shape descriptor. Contour curve-based methods simplify the leaf contour into a curve and apply various curve geometry analysis methods to describe the contour shape. However, plant leaf shapes generally exhibit small inter-class differences and large intra-class variations. Combined with affine transformations such as translation, rotation, and scaling of leaf images, leaf shape recognition faces significant challenges. In recent years, with the development of deep learning technology, end-to-end learning methods have been applied to leaf image recognition. Compared to traditional feature engineering that mimics the human eye's sensitivity to different low-level feature information, deep learning methods are entirely data-driven models built from a series of convolutional and pooling operations, better simulating the human visual nervous system and automatically extracting high-level semantic features. However, deep learning methods suffer from drawbacks such as large training sample sizes, long training times, and unclear mechanisms, and therefore cannot achieve good results in small-sample image recognition. Summary of the Invention
[0004] This invention, based on contour curve description methods, proposes a plant identification method based on the fusion of contour shape features and depth features by extracting low-level contour distance features and fusing high-level semantic features from the perspective of multi-scale and multi-feature fusion.
[0005] To address at least one of the aforementioned technical problems, according to one aspect of the present invention, a method for plant leaf image recognition based on contour shape and depth feature fusion is provided, comprising the following steps:
[0006] S10. Extract the minimum projection distance feature and the relative projection distance feature based on the blade profile curve offset characteristics, and combine them with the arch height distance feature that describes the profile convexity and concavity characteristics.
[0007] S20. Obtain multi-scale, multi-contour distance shape descriptors based on multi-feature fusion and multi-scale analysis;
[0008] S30. Use maximum value normalization to fuse low-level contour features and high-level semantic features.
[0009] Furthermore, step S10 is detailed as follows:
[0010] S11. For a binary leaf image, uniform sampling along a counterclockwise direction allows the leaf shape contour to be represented as an ordered set of points:
[0011] C = {P i (x i ,y i ),i=1,…,N}
[0012] Where N represents the number of contour sampling points;
[0013] S12. Extract the minimum projection distance; to describe the offset features of the profile curve, use the blade profile point P i Using this as a reference point, and based on the counterclockwise and clockwise directions, the arc length along the contour is s = N / 2. k+1 Contour point P i ′ and P i Using "(local distance feature reference point) as the benchmark, extract point P" i Minimum projection distance feature Its model is as follows:
[0014]
[0015]
[0016]
[0017] Among them, point Let P be the contour point. i to line P i 'P i The projection point of " Indicated by P i Let P be the starting point. i The vector whose endpoint is... and Similarly; ||P i 'P i "|| represents point P" i ′ and point P i The Euclidean distance of point P is represented by ·, where · represents the dot product of vectors; the minimum projective distance represents the distance from point P.i The degree of offset relative to one end of the contour curve;
[0018] S13. Extract relative projection distance; after extracting the minimum projection distance feature, extract point P from the perspective of describing the relative offset relationship between the two ends of the contour curve. i Relative projection distance features Its model is as follows:
[0019]
[0020] in, Represents the projection point Point P i ' the directed projection distance, Represents the projection point Time The directed projection distance;
[0021] S14. Extract the arch height distance; based on the minimum projection distance and relative projection distance, extract point P from the perspective of multi-feature fusion. i Arch height distance characteristics and Its model is as follows:
[0022]
[0023]
[0024]
[0025] Among them, h i Point P i The arch height function, where × represents a vector. with vector The vector product.
[0026] Furthermore, the outline of the middle blade is closed, P i+N =P i And P i-N =P i .
[0027] Furthermore, step S20 is detailed as follows:
[0028] Extract multi-scale contour distance features; fix the contour arc length s and contour point P i The index changes from 1 to N, and the contour distance features of all sampling points are extracted. Fixed contour point P i Change the arc length of the outline s = N / 2 k+1 Given a scale factor k, distance features at different scales k = 1, ..., T are extracted. For the maximum scale value, the model is as follows:
[0029]
[0030] in, This represents the feature vector of minimum projected distances among all contour sampling points at scale 1. Similarly.
[0031] Furthermore, step S30 is as follows:
[0032] High-level semantic features are extracted. Based on low-level distance features, a pre-trained VGG16 deep learning model based on the ImageNet image database is used, and the last fully connected layer is removed to extract high-level semantic features f of plant leaf images. d Its model is as follows:
[0033] f d =FC(I)
[0034] Where I represents the input RGB image with a size of 224×224×3, and FC(·) represents a fully connected layer with a size of 1×4096;
[0035] Fusion of features at different levels; after extracting low-level contour features and high-level semantic features, the distance D based on the low-level contour features is calculated. hf Distance D to high-level semantic features df The model that fuses features at different levels using the maximum value normalization method is as follows:
[0036]
[0037] in, and Let represent the contour feature distance or depth feature distance between the i-th image in the test set and the j-th image in the training set, respectively. and represents the maximum value of the contour feature distance sequence or depth feature distance sequence between the i-th image in the test set and all images in the training set, respectively.
[0038] Furthermore, a pre-trained VGG16 model is used to extract high-level semantic features.
[0039] Furthermore, the feature distance values at different levels range from [0,1]. By linearly fusing low-level contour features and high-level semantic features, the advantages of features at different levels complement each other.
[0040] According to another aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the plant leaf image recognition method based on contour shape and depth feature fusion of the present invention.
[0041] According to another aspect of the present invention, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the plant leaf image recognition method based on contour shape and depth feature fusion of the present invention.
[0042] Compared with the prior art, the present invention has at least the following beneficial effects:
[0043] This invention extracts minimum projection distance features and relative projection distance features from the contour curve offset angle, and combines them with the arch height distance feature describing the contour's convexity and concavity characteristics to propose a multi-scale, multi-contour distance feature blade image descriptor. This invention not only comprehensively describes the spatial distribution features of the planar contour from two orthogonal dimensions, but also possesses translation, scaling, and rotation invariance. Combining pre-trained deep learning features, a normalization strategy for low-level contour shape features and high-level semantic features is proposed, exploring the fusion mechanism of low-level contour features and high-level semantic features, significantly improving blade image recognition performance. Attached Figure Description
[0044] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings of the embodiments will be briefly described below. Obviously, the drawings described below only relate to some embodiments of the present invention and are not intended to limit the present invention.
[0045] Figure 1 This is a flowchart of the feature extraction algorithm of the present invention;
[0046] Figure 2 This is a diagram of the leaf contour extraction process in Example 1 (from left to right: original RGB image, grayscale image, binary image, contour sampling point set);
[0047] Figure 3 This is the blade profile sampling point P in Example 1. i and local distance feature reference point P i ′ and P i "Schematic diagram;"
[0048] Figure 4 This is the blade profile sampling point P in Example 1. i Schematic diagram of directional projection distance characteristics;
[0049] Figure 5This is a normalized feature diagram of Example 1 (from left to right, the images are images of different types of leaves, a normalized diagram of minimum projection distance and a normalized diagram of relative projection distance, respectively).
[0050] Figure 6 This is a schematic diagram of the normalized minimum projection distance features at 7 scales for the different types of blade outlines (256 sampling points) in Example 1 (from left to right, the scales are s = 1, 2, 4, 6 respectively).
[0051] Figure 7 This is the Swedish blade database of Example 1;
[0052] Figure 8 It is the MEW2012 blade database of Example 1. Detailed Implementation
[0053] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention.
[0054] Unless otherwise defined, the technical or scientific terms used herein shall have the ordinary meaning as understood by one of ordinary skill in the art to which this invention pertains.
[0055] like Figure 1-8 As shown,
[0056] Example 1:
[0057] Plant leaf shapes generally exhibit small inter-class differences and large intra-class variations. Combined with affine transformations of leaf images, this presents a significant challenge for leaf shape recognition. This invention first employs algorithms such as grayscale sampling models, Otsu's method, Sobel gradient operator, morphological methods, 8-connected boundary tracking, and bilinear interpolation to obtain the grayscale image, binary image, and contour point set corresponding to the original image. Then, from the perspective of leaf contour curve offset, minimum projection distance features and relative projection distance features are proposed. Furthermore, from the perspective of multi-feature fusion and multi-scale analysis, combined with the arch height distance feature describing the contour's convexity and concavity characteristics, a multi-scale, multi-contour distance shape descriptor is proposed. The proposed algorithm not only comprehensively describes contour curve features at different scales from two orthogonal dimensions but also possesses translation, scaling, and rotation invariance. Experimental results on the Swedish and MEW2012 leaf databases demonstrate the high efficiency of the proposed algorithm. Furthermore, by employing a feature distance maximum normalization strategy to fuse contour distance features and depth features, we explored the fusion mechanism of low-level contour features and high-level semantic features. Experimental results on the Swedish and MEW2012 leaf databases show that the fused features can significantly improve the performance of leaf image recognition.
[0058] To achieve the above objectives, the technical solution adopted by the present invention is as follows:
[0059] Uniform sampling of blade profile
[0060] For a binary leaf image, by uniformly sampling counterclockwise, the shape contour of the leaf can be represented as an ordered set of points:
[0061] C = {P i (x i ,y i ),i=1,…,N}
[0062] Where N represents the number of contour sampling points.
[0063] Extract minimum projection distance
[0064] To describe the offset characteristics of the profile curve, the blade profile point P is used. i Using this as a reference point, and based on the counterclockwise and clockwise directions, the arc length along the contour is s = N / 2. k+1 Contour point P i ′ and P i Using "(local distance feature reference point) as the benchmark, extract point P" i Minimum projection distance feature Its model is as follows:
[0065]
[0066]
[0067]
[0068] Among them, point Let P be the contour point. i to line P i 'P i The projection point of " Indicated by P i Let P be the starting point. i The vector whose endpoint is... and Similarly. ||P i 'P i "|| represents point P" i ′ and point P i The Euclidean distance is represented by ·, where · denotes the dot product of vectors. The minimum projected distance essentially characterizes the minimum offset of a shape contour point relative to one end of the curve.
[0069] Extract the relative projection distance;
[0070] Minimum projection distance cannot fully describe the curve offset characteristics. Based on the directed projection distance feature, a relative projection distance feature is proposed to describe the offset relationship of the contour points relative to the two ends of the curve, and to extract point P. i Relative projection distance features Its model is as follows:
[0071]
[0072] in, Represents the projection point Point P i ' the directed projection distance, Represents the projection point Point P i The directed projection distance of “”. Figure 3 and Figure 4 The profile point P of a certain blade is given respectively. i and local distance feature reference point P i 'and P i "Projection point" Directed projection distance and The diagram shows that when At time, point P i The minimum projection distance is When point P i 'and P i "Located at the projection point" When on both sides, the directional projection distance and The sign is positive, and the relative projection distance is
[0073] Extracting the arch height distance
[0074] Minimum projection distance and relative projection distance can describe the profile curve offset characteristics well, but they cannot effectively characterize the convexity and concavity characteristics of the blade profile. Based on minimum projection distance and relative projection distance, point P is extracted from the perspective of multi-feature fusion. i Arch height distance characteristics and Its model is as follows:
[0075]
[0076]
[0077]
[0078] Among them, h i Point P i The arch height function, where × represents a vector. with vector The vector product.
[0079] Extracting multi-scale contour distance features
[0080] Contour features at different scales can improve image retrieval performance. A multi-scale, multi-contour distance shape descriptor based on the number of sampling points on the contour curve is proposed. First, the contour arc length *s* is fixed, and contour points *P* are... i The index changes from 1 to N, and the contour distance features of all sampling points are extracted. Then fix the contour point P i Change the arc length of the outline s = N / 2 k+1 Given a scale factor k, distance features at different scales k = 1, ..., T are extracted. For the maximum scale value, the model is as follows:
[0081]
[0082] in, This represents the feature vector of minimum projected distances among all contour sampling points at scale 1. Similarly.
[0083] Contour distance feature normalization
[0084] An effective shape descriptor should satisfy translation, scaling, and rotation invariance. As defined, multi-scale, multi-contour distance shape descriptors based on relative distance possess inherent translation invariance but lack scaling and rotation invariance. Therefore, this paper employs maximum normalization to normalize the distance features of the same contour at the same scale, as shown in the model below.
[0085]
[0086] Figure 5 Taking scale 4 as an example, normalized minimum projection and relative projection feature maps (N=256) of different leaves a and b are given. It can be seen that although leaves a and b have a large similarity in overall shape, there is a large difference between the two distance features proposed, which have a strong class discrimination ability. Figure 6 The diagrams showing the normalized minimum projected distances of blades a and b at scales 1, 2, 4, and 6 (N = 256) are presented. As can be seen from the diagrams, smaller scales (1 and 2) focus on extracting coarse-grained contour curve features, resulting in relatively similar curves; larger scales (4 and 6) emphasize fine-grained contour curve features, leading to significant differences in the curves. By changing the length of the contour curves, a description of the contour from coarse to fine is achieved.
[0087] To eliminate the influence of contour rotation on features, Fourier transforms are performed on the proposed features of the same scale and distance, and the first M (M < N) coefficients with larger transform coefficients are selected to construct a compact, rotation-invariant multi-scale, multi-contour distance shape descriptor, the model of which is as follows:
[0088]
[0089]
[0090] Where M represents the length of the Fourier coefficients of the distance feature. and These represent the distance feature amplitudes after Fourier transform.
[0091] Shape feature similarity measurement
[0092] This invention employs a similarity measurement method based on L1 distance. The similarity model between blade shape A and shape B is as follows:
[0093]
[0094] Where w1, w2, w3, and w4 represent the weighting coefficients of the minimum projection distance l1, the relative projection distance l2, and the arch height distance l3 and l4, respectively. and l represents shape A and shape B K The distance is the nth Fourier feature at scale k.
[0095] Extracting high-level semantic features
[0096] Based on low-level distance features, a pre-trained VGG16 deep learning model based on the ImageNet image database is used, and the last fully connected layer is removed to extract high-level semantic features f of plant leaf images. d Its model is as follows:
[0097] f d =FC(I)
[0098] Where I represents the input RGB image with a size of 224×224×3, and FC(·) represents a fully connected layer with a size of 1×4096;
[0099] Deep feature similarity measurement
[0100] This invention uses L2 distance-based similarity measurement between blade images A and B, and its model is as follows:
[0101]
[0102] Integrating features at different levels
[0103] After extracting low-level contour features and high-level semantic features, the distance D based on the low-level contour features is calculated. hf Distance D to high-level semantic features df The model that fuses features at different levels using the maximum value normalization method is as follows:
[0104]
[0105] in, and Let represent the contour feature distance or depth feature distance between the i-th image in the test set and the j-th image in the training set, respectively. and represents the maximum value of the contour feature distance sequence or depth feature distance sequence between the i-th image in the test set and all images in the training set, respectively.
[0106] Based on the above, see Figure 1 A flowchart of the overall algorithm of this invention is provided. The method of this invention is as follows: First, minimum projection distance features and relative projection distance features are proposed from the perspective of blade profile curve offset. Then, from the perspective of multi-feature fusion and multi-scale analysis, combined with the camber distance feature describing the convexity and concavity characteristics of the profile, a multi-scale, multi-profile distance shape descriptor is proposed. The proposed algorithm can not only comprehensively describe the profile curve features at different scales from two orthogonal dimensions, but also has translation, scaling, and rotation invariance. Experimental results on the Swedish and MEW2012 blade databases show that the proposed algorithm is efficient. Furthermore, by using a feature distance maximum normalization strategy to fuse profile distance features and depth features, the fusion mechanism of low-level profile features and high-level semantic features is explored. Experimental results on the Swedish and MEW2012 blade databases show that the fused features can significantly improve the performance of blade image recognition.
[0107] The Swedish database contains 15 leaf types, with 75 images for each type, totaling 1125 images. The MEW2012 database contains 153 types of leaf samples, totaling 9745 samples, with 50-99 samples per type. Figure 3 and Figure 4 Sample images of each leaf type from the Swedish and MEW2012 databases are provided. As can be seen from the images, many samples exhibit significant similarity in shape characteristics, posing a considerable challenge to accurate differentiation.
[0108] The parameters of this algorithm are set as follows: the number of contour sampling points N is 1024, the number of scales T is 9, the number of retained Fourier coefficients M is 4, and the weight coefficients w1, w2, w3 and w4 of the four distance features are 0.6, 0.3, 1 and 0.4 respectively.
[0109] 1. Swedish Blade Database
[0110] (1) Classification performance:
[0111] Plant leaf recognition experiments consisted of two types: classification and retrieval. In the classification experiment, 25 images from each class in the Swedish dataset were randomly assigned as training samples, and the remaining 50 were assigned as test samples. The similarity between each sample in the test set and each image in the training set was calculated, and a nearest neighbor classifier (1-NN) was used for classification. A correct classification was achieved if the leaf image in the test sample and the most similar leaf image in the training set belonged to the same class. The accuracy rate of all test samples was recorded as the classification rate. The experiment was repeated 100 times, and the average score was taken as the final classification rate.
[0112] Table 1 shows the classification results of different algorithms on the Swedish leaf database. As can be seen from the table, the MMD classification rate is 97.92%, higher than other algorithms, indicating that the proposed algorithm has the best classification performance. Table 2 shows the classification results on the Swedish database after feature fusion at different levels under the same classification experimental method. As can be seen from the table, after feature fusion, the classification rate is 99.14%, showing a significant improvement in classification performance.
[0113] Table 1 Classification rates of various methods on the Swedish database
[0114]
[0115] Note: * indicates data derived from publicly available literature reports.
[0116] Table 2 Classification rates of various methods on the Swedish database
[0117]
[0118]
[0119] Note: * indicates data derived from publicly available literature reports.
[0120] (2) Search performance:
[0121] In the retrieval experiment, Mean Average Precision (MAP) was used to evaluate retrieval performance. The mean precision model is shown below:
[0122]
[0123]
[0124] Where AP(q) represents the average retrieval accuracy of the q-th query, Q represents the number of images in the experimental database, N represents the number of images belonging to the same class as the query image, p(k) represents the accuracy of the first k best matches, and f(k) is a labeling function, which is 1 if the k-th best match belongs to the same class as the query image, and 0 otherwise.
[0125] Table 3 shows the average precision (MAP) of the proposed algorithm on the MEW2012 leaf database, comparing it with the benchmark algorithms from the previous experiments. As shown in the table, the average precision of the MMD descriptor is 60.57%. Table 4 shows the MAP retrieval results of the MMD fused features and the benchmark algorithms on the MEW2012 leaf database. As shown in the table, the MAPs of the fused AlexNet+MMD and VGG16+MMD descriptors are 72.68% and 72.50%, respectively, which are 12.11% and 11.93% higher than the MMD descriptor. In summary, the experiments demonstrate that fusing low-level contour features with high-level semantic features can effectively characterize leaf image features and significantly improve leaf image recognition performance.
[0126] Table 3. Average accuracy of various methods on the MEW2012 database.
[0127]
[0128] Note: * indicates data derived from publicly available literature reports.
[0129] Table 4. Average accuracy of various methods on the MEW2012 database.
[0130]
[0131] Note: * indicates data derived from publicly available literature reports.
[0132] The descriptor proposed in this invention extracts minimum projection distance features and relative projection distance features to characterize contour offset characteristics, and fuses camber height distance features to represent the convexity and concavity characteristics of the contour. This descriptor can extract distance features from the orthogonal dimensions of the plane, comprehensively characterizing the contour curve features. By changing the curve length, a coarse-to-fine description is achieved. Experimental results on the Swedish and MEW2012 blade databases show that the proposed algorithm significantly outperforms existing effective contour description algorithms. To further improve blade image recognition performance, low-level contour features and high-level semantic features are fused, exploring the fusion mechanism of low-level features and high-level word features. Experimental results on the Swedish and MEW2012 blade databases show that the fused features can more effectively characterize blade image features, significantly improving blade image recognition performance.
[0133] Example 2:
[0134] The computer-readable storage medium of this embodiment stores a computer program that, when executed by a processor, implements the steps in the plant leaf image recognition method based on contour shape and depth feature fusion of Embodiment 1.
[0135] The computer-readable storage medium in this embodiment can be an internal storage unit of the terminal, such as the terminal's hard disk or memory; the computer-readable storage medium in this embodiment can also be an external storage device of the terminal, such as a plug-in hard disk, smart memory card, secure digital card, flash memory card, etc. equipped on the terminal; furthermore, the computer-readable storage medium can include both the terminal's internal storage unit and external storage devices.
[0136] The computer-readable storage medium of this embodiment is used to store computer programs and other programs and data required by the terminal. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.
[0137] Example 3:
[0138] The computer device of this embodiment includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the plant leaf image recognition method based on contour shape and depth feature fusion of Embodiment 1.
[0139] In this embodiment, the processor can be a central processing unit, or other general-purpose processors, digital signal processors, application-specific integrated circuits, off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc. The memory can include read-only memory and random access memory, and provides instructions and data to the processor. A portion of the memory can also include non-volatile random access memory. For example, the memory can also store device type information.
[0140] Those skilled in the art will understand that the content disclosed in the embodiments can be provided as a method, system, or computer program product. Therefore, this solution can take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this solution can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage) containing computer-usable program code.
[0141] This solution is described with reference to flowchart illustrations and / or block diagrams of methods and computer program products according to embodiments of this solution. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0142] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0143] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0144] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.
[0145] The examples described herein are merely preferred embodiments of the invention and are not intended to limit the concept and scope of the invention. Any modifications and improvements made by those skilled in the art to the technical solutions of the invention without departing from the design concept of the invention should fall within the protection scope of the invention.
Claims
1. A method for plant leaf image recognition based on contour shape and depth feature fusion, characterized in that, Includes the following steps: S10. Extract the minimum projection distance feature and the relative projection distance feature based on the blade profile curve offset characteristics, and combine them with the arch height distance feature that describes the profile convexity and concavity characteristics. S20. Obtain multi-scale, multi-contour distance shape descriptors based on multi-feature fusion and multi-scale analysis; S30. Use maximum value normalization to fuse low-level contour features and high-level semantic features; Step S10 is as follows: S11. For a binary leaf image, uniform sampling along a counterclockwise direction allows the leaf shape contour to be represented as an ordered set of points: ;in, Indicates the number of contour sampling points; S12. Extract the minimum projection distance; to describe the offset features of the profile curve, use the blade profile points... Using this as a reference point, based on the counterclockwise and clockwise directions, along the contour arc length is... Contour points and Using (local distance feature reference point) as a benchmark, extract points Minimum projection distance feature Its model is as follows: ; ; Among them, point For contour points to the straight line The projection point, Indicates Starting from, The vector whose endpoint is... , and Similarly; Point With point European distance, Represents the dot product of vectors; minimum projective distance represents the point. The degree of offset relative to one end of the contour curve; S13. Extract relative projection distance; after extracting the minimum projection distance feature, extract points from the perspective of the relative offset relationship between the two ends of the contour curve. Relative projection distance features Its model is as follows: ;in, Represents the projection point Time The directed projection distance, Represents the projection point Time The directed projection distance; S14. Extract the arch height distance; based on the minimum projection distance and relative projection distance, extract the points from the perspective of multi-feature fusion. Arch height distance characteristics and Its model is as follows: ; ; ;in, Point The arch height function, Representing vectors with vector The vector product; Step S20 is as follows: Extract multi-scale contour distance features; fix contour arc length Contour points The index changes from 1 to Extract the contour distance features of all sampling points. Fixed contour points Change the arc length of the outline scale factor Extracting different scales Distance features, where For the maximum scale value, the model is as follows: ;in, This represents the feature vector of minimum projected distances among all contour sampling points at scale 1. , , … , , , Similarly; Step S30 is as follows: High-level semantic features are extracted. Based on low-level distance features, a pre-trained VGG16 deep learning model based on the ImageNet image database is used, and the last fully connected layer is removed to extract high-level semantic features from plant leaf images. Its model is as follows: ;in, This represents the input RGB image, with a size of 224×224×3. This represents a fully connected layer with a size of 1×4096; Integrate features from different levels; after extracting low-level contour features and high-level semantic features, calculate the distance based on the low-level contour features. Distance to high-level semantic features The model that fuses features at different levels using the maximum value normalization method is as follows: ;in, and Let represent the contour feature distance or depth feature distance between the i-th image in the test set and the j-th image in the training set, respectively. and represents the maximum value of the contour feature distance sequence or depth feature distance sequence between the i-th image in the test set and all images in the training set, respectively.
2. The method according to claim 1, characterized in that, S11, the outline of the middle blade is closed. and .
3. The method according to claim 2, characterized in that, High-level semantic features are extracted using a pre-trained VGG16 model.
4. The method according to claim 2, characterized in that, The feature distance values at different levels range from [0,1]. By linearly fusing low-level contour features and high-level semantic features, the advantages of features at different levels complement each other.
5. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by the processor, it implements the steps in the plant leaf image recognition method based on contour shape and depth feature fusion as described in any one of claims 1 to 4.
6. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the plant leaf image recognition method based on contour shape and depth feature fusion as described in any one of claims 1 to 4.