A double-branch coding leaf grading method and system embedded with main vein thickness

Abstract

The present invention relates to a double-branch coding leaf grading method and system embedded with main vein thickness, and relates to computer vision technology and the field of leaf grading. The present invention collects blade back image data and blade front image data through an industrial camera; inputs the blade back image data into a main vein feature extraction module to extract blade main vein thickness information; inputs the blade front image data into a double-branch coding module to extract blade high-dimensional feature information; embeds the blade main vein thickness information into the blade high-dimensional feature information to generate blade fusion feature information; inputs the blade fusion feature information into a fully connected layer to output a leaf grade prediction vector; outputs the blade main vein thickness information and the leaf grade corresponding to the index position of the maximum value in the leaf grade prediction vector. The present invention solves the problem that the grading accuracy in the prior art cannot meet the requirements for accurate acquisition and pricing of leaves, extracts powerful discriminative information, and realizes accurate grading of leaves.

Description

Technical Field

[0001] The invention relates to the technical field of computer vision and the field of leaf grading, and in particular to a double-branch coding leaf grading method and system embedded with main vein thickness. Background Art

[0002] The quality inspection of leaves mainly relies on manual grading. Professional graders usually grade leaves based on their experience and sensory observation. This traditional manual grading method has problems such as high labor cost, high labor intensity, and low work efficiency. In addition, since the grading process is affected by human subjective factors, different personnel have different grading standards due to differences in experience and sensory judgment, which makes the definition of leaf grading unclear and difficult to ensure the objectivity of grading.

[0003] With the continuous development of artificial intelligence, intelligent grading methods that use computer vision technology and image analysis algorithms to extract leaf appearance quality factors are gradually emerging. However, existing intelligent leaf grading technologies mainly focus on extracting the overall appearance characteristics of leaves, such as color, shape, texture, and lesions, through traditional image processing methods. However, when dealing with leaves with similar appearance and similar growth locations or grades, it is difficult to effectively distinguish them by relying solely on these features, resulting in reduced grading accuracy and failure to meet the demand for accurate pricing of leaves. Summary of the invention

[0004] In view of the problem in the prior art that when processing leaves with similar appearance and similar growth locations or grades, it is difficult to effectively distinguish them by relying solely on these features, resulting in reduced grading accuracy and failure to meet the demand for accurate pricing of leaves, the present invention proposes a double-branch coding leaf grading method and system embedded with main vein thickness. The present invention can effectively identify leaves with similar appearance and similar growth locations or grades, extract powerful identifying information and achieve accurate leaf grading.

[0005] On the one hand, the present invention provides a double-branch coded leaf grading method embedded with main vein thickness, characterized in that it includes:

[0006] S1. Collecting image data of the back side of the blade and image data of the front side of the blade through an industrial camera;

[0007] S2. The image data of the back of the leaf is input into the main vein feature extraction module to extract the thickness information of the main vein of the leaf; the main vein feature extraction module includes a main vein segmentation module and a main vein thinning module;

[0008] S3. Inputting the front image data of the leaf into a dual-branch encoding module to extract high-dimensional feature information of the leaf; the dual-branch encoding module comprises a main encoding module, an auxiliary encoding module and an aggregation module;

[0009] S4. embedding the leaf main vein thickness information into the leaf high-dimensional feature information to generate leaf fusion feature information;

[0010] S5. Input the leaf fusion feature information into the fully connected layer and output a leaf level prediction vector;

[0011] S6. Output the leaf main vein thickness information and the leaf grade corresponding to the index position of the maximum value in the leaf grade prediction vector.

[0012] On the other hand, the present invention also provides a double-branch coding leaf grading system with embedded main vein thickness, comprising: an image acquisition unit, which acquires the leaf back image data and the leaf front image data through an industrial camera; a leaf main vein thickness information extraction unit, which is used to process the leaf back image data and extract the leaf main vein thickness information; a leaf high-dimensional feature information extraction unit, which is used to process the leaf front image data based on a double-branch coding module and extract the leaf high-dimensional feature information; a leaf information fusion unit, which embeds the leaf main vein thickness information into the leaf high-dimensional feature information to generate the leaf fusion feature information; a leaf grade prediction unit, which inputs the leaf fusion feature information into a fully connected layer and outputs the leaf grade prediction vector; and a leaf grade output unit, which outputs the leaf main vein thickness information and the leaf grade corresponding to the index position of the maximum value in the leaf grade prediction vector.

[0013] One or more technical solutions provided by the present invention have at least the following technical effects or advantages:

[0014] The industrial camera collects the image data of the back side of the blade and the image data of the front side of the blade, realizing the continuous data collection of the blade to be detected, and providing the original visual information for the subsequent leaf grade analysis. The back side image data of the blade is input into the main vein feature extraction module to extract the thickness information of the main vein of the blade, which is used to provide guiding features for the subsequent leaf grading. The front side image data of the blade is input into the dual-branch encoding module to extract the high-dimensional feature information of the blade and intelligently encode the key information in the leaf image data. The thickness information of the main vein of the blade is embedded in the high-dimensional feature information of the blade to generate the fusion feature information of the blade, and the scalar information and high-dimensional features are effectively integrated for accurate leaf grading. The fusion feature information of the blade is input into the fully connected layer, and the leaf grade prediction vector is output to provide a basis for predicting the leaf grade. The leaf grade corresponding to the index position of the maximum value in the main vein thickness information and the leaf grade prediction vector is output to replace the traditional manual grading, and improve the objectivity, accuracy and efficiency of leaf grading.

[0015] As described above, the present invention continuously collects the front and back images of leaves in real time. First, the main vein information of the leaves is extracted through the main vein feature extraction module, including the main vein area, length and thickness; then the high-dimensional feature information of the leaves is extracted through the double-branch encoding module; finally, the main vein thickness information of the leaves and the high-dimensional feature information of the leaves are weighted and spliced ​​to generate leaf fusion feature information, and the leaf grade is accurately output. It effectively identifies leaves with similar appearance and similar growth position or grade, improves the accuracy and reliability of leaf grading, and meets the demand for accurate pricing of leaves. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A schematic flow chart of a double-branch coded leaf grading method with embedded main vein thickness provided in an embodiment of the present invention;

[0017] Figure 2 A schematic diagram of a process for extracting leaf main vein thickness information in a double-branch coding leaf grading method embedded with main vein thickness provided by an embodiment of the present invention;

[0018] Figure 3 A schematic diagram of a process for extracting high-dimensional feature information of leaves in a double-branch coding leaf grading method embedded with main vein thickness provided by an embodiment of the present invention;

[0019] Figure 4 A schematic diagram of the structure of a double-branch coded leaf grading system embedded with main vein thickness provided by an embodiment of the present invention;

[0020] Figure 5 This is a diagram of collecting blade back image data and blade front image data through an industrial camera in Example 1 of the present invention;

[0021] Figure 6 This is the main vein segmentation diagram of Example 1 of the present invention;

[0022] Figure 7 This is a main vein refinement diagram of Example 1 of the present invention;

[0023] Figure 8 This is a corresponding diagram of eight levels of leaves included in the leaf level prediction vector dimension of an embodiment of the present invention. DETAILED DESCRIPTION

[0024] The embodiment of the present invention provides a double-branch coded leaf grading method and system with embedded main vein thickness, which effectively identifies leaves with similar appearance and similar growth location or grade, extracts powerful identification information and achieves the technical effect of accurate leaf grading, thus solving the problem that the grading accuracy of the prior art cannot meet the requirements for accurate leaf purchase pricing.

[0025] Example 1: Figure 1-3As shown, a double-branch coding leaf grading method embedded with main vein thickness includes:

[0026] S1. Collect the image data of the back and front of the blade by industrial camera (see Figure 5 ); Figure 5 Figure (a) shows the image data of the front side of the blade, and Figure (b) shows the image data of the back side of the blade;

[0027] S2. The image data of the back of the leaf is input into the main vein feature extraction module to extract the thickness information of the main vein of the leaf; the main vein feature extraction module includes a main vein segmentation module and a main vein thinning module;

[0028] Specifically, extracting the leaf main vein thickness information includes:

[0029] S21. Input the leaf back image data into the main vein segmentation module to generate a main vein segmentation map (see Figure 6 ); the main vein segmentation module includes a backbone network, a hierarchical feature embedding module, a hierarchical pyramid module, and a mask terminal; the main vein segmentation map generation method includes:

[0030] S211. The backbone network uses a lightweight network, and the blade back image data is input into the backbone network to generate a backbone feature map

[0031] S212. The backbone feature map Input the hierarchical feature embedding module to generate hierarchical features

[0032] S213. The hierarchical features Input the hierarchical pyramid module to generate pyramid features

[0033] S214. The pyramid feature Input the masking terminal to generate the main vein segmentation map;

[0034] S22. Input the main vein segmentation map into the main vein refinement module to generate a main vein refinement map (see Figure 7 ); Preferably, the main vein thinning module adopts a fast thinning algorithm: the fast thinning algorithm adopts a parallel thinning algorithm based on 8 neighborhoods, and determines whether the target pixel is deleted by performing step-by-step arithmetic logic operations on the 8 neighborhoods of the target pixel. The thinning process is as follows:

[0035] S221.Set X 1 is the center point, X i (i=2,3,…,9) is X 1 8 neighboring points of ;

[0036] S222. During the first iteration, if X 1 The pixel value satisfies the following formula conditions, X 1 The pixel value is set to 0:

[0037]

[0038] In the formula, Ψ(X 1 ) represents X 1 The number of times the pixel value changes from 0 to 1 in a clockwise rotation in the 8-neighborhood of .

[0039] S223. During the second iteration, if X 1 The pixel value satisfies the following formula conditions, X 1 The pixel value is set to 0:

[0040]

[0041] S224. Traverse all pixels and perform the above two iterations;

[0042] S23. Calculate the main vein area S based on the main vein segmentation map;

[0043] Specifically, based on Calculate the main vein area S, where Count is a conditional counting function, is the pixel value of each pixel point in the main vein segmentation map;

[0044] S24. Calculating the main vein length L based on the main vein refinement map;

[0045] Specifically, based on Calculate the main vein length L, where Count is a conditional counting function, is the pixel value of each pixel point in the main vein refinement image;

[0046] S25. According to the main vein area S and the main vein length L, the leaf main vein thickness information T is calculated by the formula T=S / L;

[0047] S3. Input the leaf front image data into a dual-branch encoding module to extract high-dimensional feature information of the leaf; the dual-branch encoding module comprises a main encoding module, an auxiliary encoding module and an aggregation module; preferably, the main encoding module mainly adopts a lightweight convolutional neural network structure, and the auxiliary encoding module mainly adopts a Transformer network structure;

[0048] Preferably, the method for extracting the high-dimensional feature information of the leaf specifically includes:

[0049] S31. Input the front image data of the blade into the main encoding module and the auxiliary encoding module respectively, and generate the main first feature Auxiliary first feature

[0050] S32. The main first feature and the auxiliary first feature Input the aggregation module to generate the first aggregate feature F 1 ;

[0051] S33. The first feature F of the aggregation 1 Input the main encoding module to generate the main second feature The auxiliary first feature Input the auxiliary encoding module to generate the auxiliary second feature

[0052] S34. The main second feature and the auxiliary second feature Input the aggregation module to generate the aggregated second feature F 2 ;

[0053] S35. The second feature F of the aggregation 2 Input the main encoding module to generate the main third feature The auxiliary second feature Input the auxiliary encoding module to generate the auxiliary third feature

[0054] S36. The main third feature and the auxiliary third feature Input the aggregation module to generate the third aggregate feature F 3 ;

[0055] S37. The third feature F of the aggregation 3 ; Input the main encoding module to generate the main fourth feature The auxiliary third feature Input the auxiliary encoding module to generate the auxiliary fourth feature

[0056] S38. The fourth feature of the main and the auxiliary fourth feature Input the aggregation module to generate the fourth aggregate feature F 4 , the fourth feature F 4 is the high-dimensional feature information of the leaf;

[0057] S4. embedding the leaf main vein thickness information into the leaf high-dimensional feature information to generate leaf fusion feature information; preferably, the aggregation module calculation formula is:

[0058]

[0059] In the formula, f() represents the aggregation operation, which is composed of global pooling, average pooling and adaptive pooling. The i subscript represents the i-th feature;

[0060] By formula The main vein thickness information T and the leaf high-dimensional feature information F 4 Perform weighted splicing to generate the leaf fusion feature information Q;

[0061] S5. Input the leaf fusion feature information into the fully connected layer and output a leaf level prediction vector;

[0062] Specifically, based on v=FC(Q), where FC() is a fully connected layer, and v is the leaf level prediction vector;

[0063] The leaf level prediction vector dimension includes 8 levels of leaves, specifically B1F, B2F, B3F, C2F, C3F, C4F, X2F, X3F8 levels (see Figure 8 );

[0064] S6. Output the leaf main vein thickness information and the leaf grade corresponding to the index position of the maximum value in the leaf grade prediction vector.

[0065] Example 2: Figure 4 As shown, this embodiment provides a dual-branch coded leaf grading system embedded with main vein thickness, which is used to execute a dual-branch coded leaf grading method embedded with main vein thickness, including:

[0066] An image acquisition unit, wherein the image acquisition unit acquires image data of the back side of the blade and image data of the front side of the blade through an industrial camera;

[0067] A leaf main vein thickness information extraction unit, the leaf main vein thickness information extraction unit is used to process the leaf back image data to extract the leaf main vein thickness information;

[0068] A leaf high-dimensional feature information extraction unit, the leaf high-dimensional feature information extraction unit is used to process the leaf front image data based on a double-branch encoding module to extract the leaf high-dimensional feature information;

[0069] A leaf information fusion unit, wherein the leaf information fusion unit embeds the leaf main vein thickness information into the leaf high-dimensional feature information to generate the leaf fusion feature information;

[0070] A leaf grade prediction unit, wherein the leaf grade prediction unit inputs the leaf fusion feature information into a fully connected layer and outputs the leaf grade prediction vector;

[0071] A leaf grade output unit, wherein the leaf grade output unit outputs the leaf main vein thickness information and the leaf grade corresponding to the index position of the maximum value in the leaf grade prediction vector.

[0072] Through the above-mentioned detailed description of a double-branch coded leaf grading method embedded with main vein thickness in this specification, those skilled in the art can clearly know that a double-branch coded leaf grading system embedded with main vein thickness in this embodiment is a system. For the system disclosed in Example 2, since it corresponds to the method disclosed in Example 1 and has corresponding functional modules and beneficial effects, the relevant parts can be referred to the description of the method part.

[0073] The specific implementation modes of the present invention are described in detail above, but the present invention is not limited to the above implementation modes, and various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention.

Claims

1. A double-branch coded leaf grading method embedded with main vein thickness, characterized in that: include: S1. Collecting image data of the back side of the blade and image data of the front side of the blade through an industrial camera; S2. The image data of the back of the leaf is input into the main vein feature extraction module to extract the thickness information of the main vein of the leaf; the main vein feature extraction module includes a main vein segmentation module and a main vein thinning module; S3. Inputting the front image data of the leaf into a dual-branch encoding module to extract high-dimensional feature information of the leaf; The dual-branch encoding module includes a main encoding module, an auxiliary encoding module and an aggregation module; S4. embedding the leaf main vein thickness information into the leaf high-dimensional feature information to generate leaf fusion feature information; S5. Input the leaf fusion feature information into the fully connected layer and output a leaf level prediction vector; S6. Output the leaf main vein thickness information and the leaf grade corresponding to the index position of the maximum value in the leaf grade prediction vector.

2. The double-branch coded leaf grading method with embedded main vein thickness according to claim 1 is characterized in that: The S2. extracting the leaf main vein thickness information specifically includes: Inputting the image data of the back side of the leaf into the main vein segmentation module to generate a main vein segmentation map; Inputting the main vein segmentation map into the main vein thinning module to generate a main vein thinning map; Calculate the main vein area based on the main vein segmentation map ; Calculate the main vein length based on the main vein refinement map ; According to the main vein area and the main vein length , through the formula Calculate the thickness of the main vein of the leaf .

3. The double-branch coded leaf grading method with embedded main vein thickness according to claim 2 is characterized in that: The main vein segmentation module includes a backbone network, a hierarchical feature embedding module, a hierarchical pyramid module, and a mask terminal. The method further includes: The backbone network uses a lightweight network, and the blade back image data is input into the backbone network to generate a backbone feature map ; The backbone feature map Input the hierarchical feature embedding module to generate hierarchical features ; The hierarchical features Input the hierarchical pyramid module to generate pyramid features ; The pyramid feature Input the mask terminal to generate the main vein segmentation map.

4. The double-branch coded leaf grading method with embedded main vein thickness according to claim 2 is characterized in that: The main vein thinning module adopts a fast thinning algorithm, and inputs the main vein segmentation map into the fast thinning algorithm to generate the main vein thinning map.

5. The double-branch coded leaf grading method with embedded main vein thickness according to claim 1 is characterized in that: The S3. extracting the high-dimensional feature information of the leaf specifically includes: The front image data of the blade is input into the main encoding module and the auxiliary encoding module respectively to generate the main first feature , Auxiliary first feature ; The main first feature and the auxiliary first feature Input the aggregation module to generate the first aggregation feature ; The first feature of the aggregation Input the main encoding module to generate the main second feature , the auxiliary first feature Input the auxiliary encoding module to generate the auxiliary second feature ; The main second feature and the auxiliary second feature Input the aggregation module to generate the second aggregation feature ; The second feature of the aggregation Input the main encoding module to generate the main third feature , the auxiliary second feature Input the auxiliary encoding module to generate the auxiliary third feature ; The main third feature and the auxiliary third feature Input the aggregation module to generate the aggregated third feature ; The third feature of the aggregation ; Input the main encoding module to generate the main fourth feature , the auxiliary third feature Input the auxiliary encoding module to generate the auxiliary fourth feature ; The fourth feature of the main and the auxiliary fourth feature Input the aggregation module to generate the fourth feature of aggregation , the fourth feature is the high-dimensional feature information of the leaf.

6. The double-branch coded leaf grading method with embedded main vein thickness according to claim 1, characterized in that: The main encoding module adopts a lightweight convolutional neural network structure, and the auxiliary encoding module adopts a Transformer network structure.

7. The double-branch coded leaf grading method with embedded main vein thickness according to claim 1, characterized in that: The calculation formula of the aggregation module is: ； In the formula, Represents the aggregation operation, which consists of global pooling, average pooling and adaptive pooling. Subscript represents the feature.

8. The double-branch coded leaf grading method with embedded main vein thickness according to claim 1, characterized in that: S4. Method for generating leaf fusion feature information: by formula The main vein thickness information And the high-dimensional feature information of the leaves Perform weighted splicing to generate the leaf fusion feature information .

9. A double-branch coded leaf grading system embedded with main vein thickness, characterized in that: A double-branch coded leaf grading method for implementing any one of claims 1 to 8 with embedded main vein thickness, comprising: An image acquisition unit, wherein the image acquisition unit acquires image data of the back side of the blade and image data of the front side of the blade through an industrial camera; A leaf main vein thickness information extraction unit, the leaf main vein thickness information extraction unit is used to process the leaf back image data to extract the leaf main vein thickness information; A leaf high-dimensional feature information extraction unit, the leaf high-dimensional feature information extraction unit is used to process the leaf front image data based on a double-branch encoding module to extract the leaf high-dimensional feature information; A leaf information fusion unit, wherein the leaf information fusion unit embeds the leaf main vein thickness information into the leaf high-dimensional feature information to generate the leaf fusion feature information; A leaf grade prediction unit, wherein the leaf grade prediction unit inputs the leaf fusion feature information into a fully connected layer and outputs the leaf grade prediction vector; A leaf grade output unit, wherein the leaf grade output unit outputs the leaf main vein thickness information and the leaf grade corresponding to the index position of the maximum value in the leaf grade prediction vector.

Images