A seed feature extraction and classification recognition system and method

Abstract

The application discloses a kind of seed feature extraction and classification identification system and method, the method therein includes light box, the top of the light box is equipped with camera, the camera is micro lens installed on mobile phone, mobile phone is equipped with built-in memory card;The two sides of the camera are equipped with light source respectively;The inner bottom surface of the light box is equipped with object table, for placing plant seed physical object, the object table is contacted with seed surface and is paved with a layer of black velvet.The system and method of the application can greatly reduce the cost of seed classification and other scientific research tasks, improve efficiency, promote the development of agricultural intelligence and modernization.

Description

Technical Field

[0001] This invention relates to seed identification technology, specifically to a seed feature extraction and classification identification system and method. Background Technology

[0002] Currently, weeds are one of the three major pests in agricultural and forestry production, causing serious harm to agricultural production, the ecological environment, and biodiversity. Seed purity analysis, i.e., testing for normal seeds, requires seed classification and identification. Rapid and accurate classification and identification of weed seeds can, to a certain extent, prohibit and limit weeds at their source, preventing and reducing weed damage, and is an effective means of protecting agricultural and forestry economic security. Furthermore, seed classification is indispensable in the processes of seed production, sales, storage, and sowing.

[0003] Traditional seed classification and identification methods primarily involve expert participation, using microscopes or magnifying glasses for visual inspection. This approach is characterized by high costs and slow speed. Physical methods, which classify seeds based on size and weight, require specialized equipment and have relatively poor screening results. Chemical and biological methods, using RNA and DNA to identify seeds, are also costly and not suitable for widespread adoption.

[0004] Existing technical solutions:

[0005] One invention provides an air-separation seed screening device, comprising a worktable, a screening plate inclinedly disposed on the worktable, air holes evenly distributed on the screening plate, a feeding auger disposed on the worktable and above the screening plate, an air supply module disposed on the worktable and on the non-working side of the screening plate, and a recovery tank disposed on the worktable and below the screening plate. The feeding auger has multiple output ends, which are spaced apart above the screening plate.

[0006] A simplified method for identifying the purity of *Vaccaria spicata* varieties using SCoT molecular markers has been provided. Its main features include the following steps: (1) extraction of *Vaccaria spicata* germplasm DNA; (2) PCR amplification of the *Vaccaria spicata* germplasm DNA obtained in step (1) using SCoT primers; (3) agarose gel electrophoresis and photographing of the PCR amplification products; and (4) comparison of differences in polymorphic sites in *Vaccaria spicata* germplasm to determine the purity identification results. The detection method used in this invention can complete the identification of seed variety purity within 6 hours.

[0007] A method for classifying wheat seeds based on multi-scale feature extraction has been proposed, which relates to the field of hyperspectral technology. This method trains the model using a structurally improved multi-scale deep learning network structure. The three-dimensional convolutional kernels in the multi-scale deep learning network structure have at least two different scale types in the spectral dimension. Thus, each convolutional unit can be used to extract features from the sample image from the two-dimensional spatial dimension and the one-dimensional spectral dimension, fully mining the features in the hyperspectral data.

[0008] Disadvantages of existing technology:

[0009] Classification methods using gravity are ineffective and more suitable for initial selection. While chemical and biological methods offer high accuracy for seed identification, they are also costly and have limited application scenarios. Ordinary computer vision techniques for seed classification have limited capacity for identifying seed types and also exhibit low accuracy. Summary of the Invention

[0010] The main objective of this invention is to provide a seed feature extraction and classification system and method to solve the problems of small seed size, many varieties, and difficulty in data collection.

[0011] According to one aspect of the present invention, a seed feature extraction and classification system is provided, comprising a light box, a camera mounted on the top of the light box, the camera being a macro lens mounted on a mobile phone, the mobile phone having a built-in memory card; and light sources respectively mounted on both sides of the camera.

[0012] The bottom of the light box is equipped with a platform for placing plant seeds. The surface of the platform that contacts the seeds is covered with a layer of black velvet.

[0013] Furthermore, the macro lens has a magnification of 6x, 12x, or 21x respectively.

[0014] Furthermore, the lightbox is a double-layered, opaque PP plastic box with dimensions of 330mm × 260mm × 260mm.

[0015] Furthermore, the light source consists of two LED panels with a color temperature of 5500K, an input voltage of 100V to 240V, an output voltage of 12V, and a brightness of 8500LUX. The light is sufficient to cover the entire seed and the light is uniform.

[0016] According to another aspect of the present invention, a method for seed feature extraction and classification is provided, comprising:

[0017] Data acquisition and processing;

[0018] Extraction of seed features;

[0019] The establishment and training of the seed identification network.

[0020] Furthermore, the acquisition and processing of the data includes:

[0021] A data acquisition device was set up to collect 88 types of seed images. At the same time, the data was preprocessed such as cropping, and after manual annotation, screening and processing, a dataset was created to train the neural network.

[0022] After the image is acquired and stored in the device, it can be directly imported when feature extraction is performed.

[0023] Furthermore, the extraction of the seed features includes:

[0024] To extract the following seed image features:

[0025] Basic characteristics: area, perimeter, centroid, major axis, minor axis, ratio of major to minor axis, maximum span, maximum inscribed circle radius, minimum circumscribed circle radius;

[0026] Color characteristics: mean and variance of R, G, B, and L;

[0027] Invariant moment characteristics: 7 Hu invariant moments;

[0028] Texture features: LBP, Uniform LBP features;

[0029] The original LBP operator is defined as follows: within a 3*3 window, using the center pixel as a threshold, the gray values ​​of the 8 adjacent pixels are compared with the center pixel. If the value of the surrounding pixels is greater than the value of the center pixel, the position of the pixel is marked as 1; otherwise, it is marked as 0. In this way, the comparison of the 8 points in the 3*3 neighborhood can generate an 8-bit binary number. After being converted to decimal, it becomes the LBP value of the center pixel in this neighborhood. By calculating the LBP values ​​of all pixels in the entire image in this way, the LBP feature map reflecting the local texture features of the image is obtained.

[0030] An LBP operator produces different binary patterns. For an LBP operator with P sampling points within a circular region of radius R, it will produce 2... P Types of patterns;

[0031] When the cyclic binary number corresponding to a certain LBP changes from 0 to 1 or from 1 to 0 at most twice, the binary number corresponding to that LBP is called an equivalent pattern class.

[0032] Furthermore, the establishment and training of the seed identification network includes:

[0033] The self-attention mechanism in Transformer is used to obtain information between each block;

[0034] After screening, blocks with significant discriminative regions are selected and processed by CNN;

[0035] The Transformer structure consists of several blocks. The weight vector returned by each attention head in each Transformer block is fed into the partial selection module for computation. Let the multi-head attention structure have a total of... Each attention point, then The weight vector returned by each block It can be represented as:

[0036] (1)

[0037] (2)

[0038] After obtaining After calculating the weight vectors of each layer, multiplying them together yields the encoding vector that captures information propagating from the input layer to higher layers. :

[0039] (3)

[0040] against Sort each element in the array and take the first few elements. The index of each element; then select the partial selection module PSM, filtering from it. vectors Using the same method, the image selection module (ISM) is used to extract the image from the input raw image. block ;

[0041] These two modules are used to filter out vectors with high information content and salient regions in images;

[0042] vector go through Each Encoder module performs encoding and returns the weight matrix 'a' for each layer and the output of the Encoder. Then, PSM selects from them to obtain the selected results. Similarly, the original image is obtained after selection by the ISP. ;

[0043] Then The data is fed into a CNN for processing to obtain local information about the region. Then and Make a connection and send it into the block for processing to obtain the result. Specifically, it can be expressed as:

[0044] , ,

[0045]

[0046] The encoded vector and local information are combined and fed into the block. Features with long-range information and features with local information obtained through CNN processing are combined and then output.

[0047] Advantages of this invention:

[0048] This invention first establishes a data acquisition device that is low-cost and easy to operate. Through this device, a seed image dataset is acquired. This dataset, being a rare type of seed image dataset, can not only be used for training seed recognition models but also provide a large amount of image data for seed research.

[0049] This invention employs a novel seed image recognition method combining Transformer and CNN (Convolutional Neural Network), which effectively improves seed recognition accuracy. Furthermore, this invention can be easily operated via a desktop system. The system features seed feature extraction, including seed color and area; seed classification; and, after seed recognition, can display relevant encyclopedic information about the seed.

[0050] This invention can greatly reduce the cost of scientific research tasks such as seed classification, improve efficiency, and promote the intelligent and modern development of agriculture.

[0051] The data acquisition device of this invention enables simple image data acquisition, and through computer vision technology, seed classification and feature extraction can be achieved from seed images. This method is low-cost, easy to use, and significantly reduces manpower and material resources, improving both efficiency and accuracy. It is of great significance for promoting the intelligent and modern development of agriculture.

[0052] In addition to the objectives, features, and advantages described above, the present invention has other objectives, features, and advantages. The invention will now be described in further detail with reference to the figures. Attached Figure Description

[0053] The accompanying drawings, which form part of this application, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0054] Figure 1This is a schematic diagram of the image acquisition device of the present invention;

[0055] Figure 2 This is an example of comparing the original image acquired in this invention with the grayscale image and the LBP feature map.

[0056] Figure 3 This is the model framework of the present invention;

[0057] Figure 4 This is a functional block diagram of the present invention;

[0058] Figure 5 This is an operation flowchart of the present invention;

[0059] Figure 6 This is a diagram of the system interaction interface of the present invention.

[0060] Figure label:

[0061] 1 is the camera, 2 is the light box, 3 is the stage, 4 is the light source, and 5 is the seed. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0063] refer to Figures 1 to 6 ,

[0064] I. Data Acquisition and Processing

[0065] In image classification tasks, model training requires data support. This invention sets up a data acquisition device to collect 88 seed images. At the same time, the data is preprocessed such as cropping, and after manual annotation, screening and processing, a dataset is created to train the neural network.

[0066] Images can also be acquired using this device. After the images are acquired and stored in the device, they can be directly imported for feature extraction. To ensure the clarity of the collected images, the feature extraction system implemented in this paper designs an image acquisition device to achieve the goal of acquiring high-quality images. The structure is as follows: Figure 1 As shown. Figure 1Component 1 is a camera unit, which mounts a macro lens onto a mobile phone and has a built-in memory card. Ordinary lenses have limited focusing capabilities for small targets, so a macro lens is needed for close-up focusing on the seeds. Lens magnification options include 6x, 12x, and 21x; the appropriate lens can be selected based on the actual size of the seeds. Component 2 is a lightbox, a double-layered, opaque PP plastic box measuring 330×260×260mm. Component 3 is a stage for placing the plant seeds; a black velvet cloth is placed on the surface in contact with the seeds, resulting in a black background for the image, enhancing the color contrast between the seeds and the background. Component 4 is a light source, consisting of two 5500K color temperature LED panels, with an input voltage of 100V to 240V, an output voltage of 12V, and a brightness of 8500LUX, providing sufficient and uniform illumination to cover the entire seed. Component 5 is the seed; simply place the seed at the center mark on the stage when capturing images. The captured images range in size from 50K to 1M, with dimensions of approximately 1600×1200, meeting the experimental requirements. This image acquisition device is easy to operate and acquires high-quality images, making it suitable for seed feature extraction and laying a good foundation for the implementation of feature extraction functions.

[0067] II. Extraction of Seed Features

[0068] This system can extract the following seed image features:

[0069] Basic characteristics: area, perimeter, centroid, major axis, minor axis, ratio of major to minor axis, maximum span, maximum inscribed circle radius, minimum circumscribed circle radius.

[0070] Color characteristics: Mean and variance of R, G, B, and L

[0071] Invariant moment characteristics: 7 Hu invariant moments.

[0072] Texture features: LBP (Local Binary Pattern) and Uniform LBP features.

[0073] In this system, the two texture features are considered higher-level features, as they can reflect the texture information of the image and are of great significance to machine learning and grassland science. The following is an introduction to the two LBP texture features.

[0074] The original LBP operator is defined as follows: within a 3x3 window, using the center pixel as a threshold, the grayscale values ​​of the eight adjacent pixels are compared with the center pixel. If the value of a surrounding pixel is greater than the value of the center pixel, the pixel is marked as 1; otherwise, it is marked as 0. Thus, the comparison of the eight points within the 3x3 neighborhood produces an 8-bit binary number. Converted to decimal, this becomes the LBP value of the center pixel within that neighborhood. Calculating the LBP values ​​of all pixels in the entire image using this method yields an LBP feature map reflecting the local texture features of the image. A comparison is shown in the image below. Figure 2 As shown.

[0075] An LBP operator can generate different binary modes. For an LBP operator with P sampling points within a circular region of radius R, it will generate 2... P To address the problem of excessive binary patterns and improve statistical accuracy, Ojala proposed using a "Uniform Pattern" to reduce the dimensionality of LBP operator pattern types. An "equivalent pattern" is defined as follows: if the cyclic binary number corresponding to an LBP transitions from 0 to 1 or from 1 to 0 at most two times, then the binary representation of that LBP is called an equivalent pattern class. Patterns other than those in the equivalent pattern class are classified into another category, called the hybrid pattern class.

[0076] This improvement significantly reduces the number of binary modes without losing any information. The number of modes is reduced from the original P. 2 The number of binary patterns is reduced to 59, where P represents the number of sampling points in the neighborhood set. For 8 sampling points in a 3×3 neighborhood, the number of binary patterns is reduced from the original 256 to 59. This results in a smaller feature vector dimension and reduces the impact of high-frequency noise.

[0077] III. Establishment and Training of Seed Recognition Network

[0078] A seed recognition network based on TransFG and ResNet is proposed. Experiments revealed that each part of a seed possesses specific texture and other features, which are crucial for seed classification and recognition. However, processing seed images in blocks leads to a lack of connection between blocks and destroys features such as seed shape. Therefore, this invention utilizes the self-attention mechanism in Transformer to acquire information between blocks. However, Transformer cannot effectively utilize the local information of small blocks; therefore, CNN is used to address this issue. Inspired by TransFG, instead of processing the entire image into the CNN, blocks with salient discriminative regions are selected and processed by the CNN.

[0079] The Transformer structure in this experiment consists of several blocks (Encoders). The weight vector returned by each attention head in each Transformer block is fed into the partial selection module for computation. Assuming the multi-head attention structure has K attention heads, then... The weight vector returned by each block It can be represented as:

[0080] (1)

[0081] (2)

[0082] After obtaining After calculating the weight vectors of each layer, multiplying them together yields the encoding vector that captures information propagating from the input layer to higher layers. :

[0083] (3)

[0084] Because the original attention weights do not necessarily correspond to the relative importance of the inserted labels, especially for higher levels of the model, due to the lack of discernibility of the embedded labels, they are different from the original attention weights of a single layer. In comparison, the cumulative integration As a better option for selecting discriminative regions.

[0085] At this time, in response to Sort each element in the array and take the first few elements. The index of each element. Then you can partially select modules (PSM, Patch Select Module) and filter from them. vectors Using the same method, the Image Select Module (ISM) can extract images from the input raw image. block These two modules can be used to filter out vectors with high information content and salient regions in images.

[0086] The overall structure of the model is as follows Figure 3 As shown, vector After L Each Encoder module performs encoding and returns the weight matrix 'a' for each layer and the output of the Encoder. Then, PSM selects from them to obtain the selected results. Similarly, the original image is obtained after selection by the ISP (Image Select Module). Instead of performing convolution operations on all blocks of the original image, only the blocks selected by the ISP are convolved. This not only saves computational resources but also reduces interference from the background. Then... The data is fed into a CNN for processing to obtain local information about the region. Then and Make a connection and send it into the block for processing to obtain the result. Specifically, it can be expressed as:

[0087] , ,

[0088]

[0089] The encoded vector and local information are combined and fed into the block. This combines features with long-range information and features with local information obtained through CNN processing, and then outputs them. This avoids the CNN processing global information.

[0090] The user interface of this system mainly includes image segmentation, image preprocessing, feature extraction, image recognition, and others. Functional modules include... Figure 4 .

[0091] The File menu primarily includes functions for importing images and image segmentation. The shortcut keys are "Ctrl+F" and "Ctrl+P" respectively. To import an image, select the image you want to import in the local file browsing window. Note that the path name must not contain Chinese characters.

[0092] Image processing – mainly includes functions for grayscale conversion, binarization, and edge extraction of images. Clicking the “Grayscale” option displays the grayscale image of the imported image in the interface. Clicking the “Binarization” option displays the binary image of the imported image in the interface. Clicking the “Edge Extraction” option will display the outline of the seed after filling and morphological processing using the binary image, and then extracting it using the Canny edge detection operator.

[0093] The feature extraction function includes two parts: general features and texture features.

[0094] General features – mainly include the extraction of basic morphological features, epidermal color features, and invariant moment features of seeds. Clicking the “Basic Features” option will display the seed area, perimeter, major and minor axes of the fitted ellipse, maximum span, and other feature values ​​in the text box in the upper right corner. Clicking the “Color Features” option will display the mean and standard deviation of the RGB components and brightness of the imported image in the text box. Clicking the “Invariant Moment Features” option will output 7 Hu invariant moment feature values ​​in the text box. General features are all output in numerical form.

[0095] Texture features – mainly include the extraction of LBP features and Uniform LBP features from the seed. Clicking the “LBP_1” and “LBP_2” options will display the feature maps of LBP and Uniform LBP in the interface, and the corresponding feature histograms will be displayed in the pop-up window.

[0096] Image Recognition – Click the “Start Recognition” option to perform seed recognition. The recognition results and corresponding confidence scores are displayed in the interface in data form, with a total of 5 records output.

[0097] Other – mainly includes the user manual and instructions about the system, which are used to help users understand how to use the system and developer information, all displayed in pop-up windows.

[0098] User operation process as follows Figure 5 The system interface is as follows Figure 6 As shown.

[0099] This invention uses a simple image acquisition device to acquire seed image data and establish a seed dataset.

[0100] This invention proposes a novel network model for seed image recognition based on the combination of Transformer and CNN (Convolutional Neural Network). This model can accurately and quickly identify seeds, and is low in cost and easy to operate.

[0101] The desktop system of this invention can extract seed features, including color, shape, and area. It also enables seed classification and identification, providing a plant encyclopedia, including information on the root, stem, leaves, and habitat of the corresponding plant.

[0102] In the image acquisition device of the present invention, the shooting device can be a mobile phone, camera, professional equipment, etc., and the image can also be changed from an RGB image to a hyperspectral image.

[0103] The desktop system of this invention can be further optimized, with added advanced features and an improved interface. It can also be used on mobile devices such as smartphones.

[0104] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for seed feature extraction and classification, characterized in that, include: Data acquisition and processing; Extraction of seed features; The establishment and training of the seed identification network; The acquisition and processing of the data includes: A data acquisition device was set up to collect 88 types of seed images. The data was cropped and preprocessed, and then manually labeled, filtered and processed to create a dataset for training neural networks. After the image is acquired and stored in the device, it can be directly imported when feature extraction is performed. The extraction of seed features includes: To extract the following seed image features: Basic characteristics: area, perimeter, centroid, major axis, minor axis, ratio of major to minor axis, maximum span, maximum inscribed circle radius, minimum circumscribed circle radius; Color characteristics: mean and variance of R, G, B, and L; Invariant moment characteristics: 7 Hu invariant moments; Texture features: LBP, Uniform LBP features; The original LBP operator is defined as follows: within a 3*3 window, using the center pixel as a threshold, the gray values ​​of the 8 adjacent pixels are compared with the center pixel. If the value of the surrounding pixels is greater than the value of the center pixel, the position of the pixel is marked as 1; otherwise, it is marked as 0. In this way, the comparison of the 8 points in the 3*3 neighborhood can generate an 8-bit binary number. After being converted to decimal, it becomes the LBP value of the center pixel in this neighborhood. By calculating the LBP values ​​of all pixels in the entire image in this way, the LBP feature map reflecting the local texture features of the image is obtained. An LBP operator produces different binary patterns. For an LBP operator with P sampling points within a circular region of radius R, it will produce 2... P Types of patterns; When the cyclic binary number corresponding to a certain LBP changes from 0 to 1 or from 1 to 0 at most twice, the binary number corresponding to that LBP is called an equivalent pattern class. The establishment and training of the seed identification network includes: The self-attention mechanism in Transformer is used to obtain information between each block; After screening, blocks with significant discriminative regions are selected and processed by CNN; The Transformer structure consists of several blocks. The weight vector returned by each attention head in each Transformer block is fed into the partial selection module for computation. Let the multi-head attention structure have K attention heads in total. The weight vector returned by each block It can be represented as: (1) (2) After obtaining After calculating the weight vectors of each layer, multiplying them together yields the encoding vector that captures information propagating from the input layer to higher layers. : (3) against Sort each element in the array and take the first few elements. The index of each element; then select the partial selection module PSM, filtering from it. vectors Using the same method, the image selection module (ISM) is used to extract the image from the input raw image. block ; These two modules are used to filter out vectors with high information content and salient regions in images; Filter out vectors with more information go through Each Encoder module performs encoding and returns the weight matrix 'a' for each layer and the output of the Encoder. Then, PSM selects from them to obtain the selected results. Similarly, the original image is obtained after selection by the ISP. ; Then The data is fed into a CNN for processing to obtain local information about the region. Then and Make a connection and send it into the block for processing to obtain the result. Specifically, it means: , , ， The encoded vector and local information are combined and fed into the block. Features with long-range information and features with local information obtained through CNN processing are combined and then output.

2. An apparatus for applying the seed feature extraction and classification method according to claim 1 The device is characterized by, Includes a light box (2), the top of which is equipped with a camera (1), the camera (1) being a macro lens mounted on a mobile phone, the mobile phone having a built-in memory card; light sources (4) are respectively provided on both sides of the camera (1). The bottom surface of the light box (2) is provided with a platform (3) for placing plant seeds. The surface of the platform (3) in contact with the seeds is covered with a layer of black velvet. The macro lenses have magnifications of 6x, 12x, or 21x respectively; The light box (2) is a double-layered opaque PP plastic box with dimensions of 330mm×260mm×260mm; The light source (4) consists of two LED panels with a color temperature of 5500K, an input voltage of 100V to 240V, an output voltage of 12V, a brightness of 8500LUX, and sufficient light to cover the entire seed, with uniform light distribution.

Images