Wheat variety identification method and system based on linear array camera and near infrared

Abstract

The invention discloses a wheat variety identification method and system based on a linear array camera and near infrared rays, relates to the technical field of agricultural product quality detection and automation, and aims to solve the problem that high-speed online identification of wheat varieties cannot be realized on the premise of ensuring high precision in the prior art. Respectively acquiring image data and near infrared spectrum data of the same grain in the same detection window of the free falling path of the grain; s2, preprocessing the spectral data in an edge computing device, and extracting chemical component characteristics by adopting a Transform self-attention mechanism; s3, preprocessing the image data, and extracting image features of the grains; s4, taking the image features as keys and the spectral features as queries by using a cross attention mechanism, calculating similarity and fusing to obtain a fused feature vector; and S5, inputting the fusion feature vector into a lightweight model optimized by TensorRT, completing variety classification and quality index prediction at an edge computing device, and outputting a result in real time.

Description

Technical Field

[0001] This invention relates to the field of agricultural product quality testing and automation technology, specifically to a method and system for wheat variety identification based on a linear array camera and near-infrared spectroscopy. Background Technology

[0002] Online identification of wheat varieties is a core means of achieving "pricing based on quality and grading by storage and warehousing" in grain storage, processing, and breeding. Traditional manual sensory judgment is inefficient and prone to subjective errors; molecular methods such as PCR and electrophoresis, while highly accurate, require damage to grains, involve expensive reagents, and have testing cycles measured in hours, making them unsuitable for integration into grain purchasing production lines with flow rates exceeding ten tons per hour. In recent years, visible / near-infrared spectroscopy and machine vision have been introduced: spectral technology can non-destructively obtain chemical composition using the combination and harmonic signals of molecular vibrations, while visual technology acquires shape and color features through area or line array cameras. However, single spectra are affected by spot area, particle gaps, and surface contamination, resulting in poor repeatability; single images are insensitive to chemical differences, making it difficult to distinguish varieties with similar morphology but different internal components. While merging the two can improve accuracy, the commonly used "time-sharing acquisition—PC offline processing" architecture suffers from drawbacks such as large data transmission delays, reliance on servers for on-site computing power, and large model sizes, making it difficult to balance "accuracy" and "speed," and hindering on-the-fly inspection in grain storage environments with severe dust, vibration, and temperature changes.

[0003] For example, there is a Chinese patent with publication number CN120411762A, which relates to a "wheat variety identification method based on hyperspectral imaging". It uses a push-broom hyperspectral camera to acquire "image-spectral integration" data of grains in the range of 400 nm-1000 nm, establishes regional models and selects the optimal model, and then calculates the purity of blind samples. However, the Chinese patent with publication number CN120411762A has the following problems: First, the push-broom platform is used for data acquisition, and the scanning time of a single sample is measured in seconds, which cannot meet the needs of continuous flow in the grain collection line. Second, it only uses hyperspectral information and does not integrate morphological and dynamic features, resulting in a high misjudgment rate for varieties with similar bloodlines and small spectral differences. Third, all data needs to be transmitted back to the PC for model calculation, which consumes a lot of bandwidth and has high latency, making it impossible to obtain results in real time on site. Fourth, the push-broom mechanism and high-precision displacement platform have complex structures, high costs, and high maintenance requirements, making them unsuitable for long-term operation in dusty and space-constrained grain collection sites. Summary of the Invention

[0004] To overcome the problem that existing technologies cannot achieve high-speed online identification of wheat varieties while ensuring high accuracy, this invention proposes a wheat variety identification method and system based on a linear array camera and near-infrared spectroscopy. By combining a high-frequency linear array camera and near-infrared spectroscopy, high-speed online identification of wheat varieties can be achieved without reducing identification accuracy.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: a method for wheat variety identification based on a linear array camera and near-infrared spectroscopy, comprising: S1, within the same detection window along the free fall path of the grain, acquire image data and near-infrared spectral data of the same grain respectively; S2, the spectral data is preprocessed within the edge computing device, and chemical composition features are extracted using the Transformer self-attention mechanism; S3, preprocess the image data to extract the image features of the seeds; S4 uses a cross-attention mechanism to use image features as keys and spectral features as queries, calculates similarity and fuses them to obtain a fused feature vector. S5 inputs the fused feature vectors into a lightweight model optimized by TensorRT, completes variety classification and quality index prediction on an edge computing device, and outputs the results in real time.

[0006] In this technical solution, the morphological image of the grain and the near-infrared spectrum are simultaneously acquired at the microsecond level within the same detection window, and the two types of information are complemented and fused in real time at the edge. Thus, without reducing the identification accuracy, the remote server and complex motion mechanism are eliminated, and high-speed online identification of wheat varieties can be achieved by detecting them as they flow, solving the core pain point that existing technologies cannot balance accuracy and speed.

[0007] Preferably, in step S4, the cross-attention mechanism includes: spectral features encoded by Transformer as queries, image features encoded by CNN as keys, weights calculated by scaling dot product attention, and appearance information weighted into chemical features to form a fused feature vector.

[0008] Preferably, in step S5, the TensorRT optimized model compresses the original GBDT or CNN model to less than 100 MB through knowledge distillation, and the single-seed inference time on the Jetson Nano edge device does not exceed 20 ms.

[0009] Preferably, in step S1, obtaining image data and near-infrared spectral data of the same grain includes: a photoelectric trigger emitting a microsecond-level synchronization signal to trigger the pulse light source, high-frequency linear array camera, and fiber optic probe-type near-infrared spectrometer to work simultaneously, thereby obtaining image data and near-infrared spectral data of the same grain.

[0010] Preferably, in step S2, the spectral data preprocessing is performed using the standardized difference method, the Savitzky-Golay smoothing method, or detrending processing.

[0011] Preferably, in step S3, the image data preprocessing includes denoising, enhancement, binarization, and contour extraction.

[0012] Preferably, the line frequency of the high-frequency linear array camera is not less than 8000 Hz and the exposure time is not more than 10 μs.

[0013] Preferably, in step S4, the fused feature vector is obtained by interim feature fusion.

[0014] Preferably, the image features include morphological features, color features, local features, and dynamic features; the morphological features include rectangularity and surface area to volume ratio; the color features are obtained through RGB three channels; the local features include embryonic end depression, abdominal curvature, dorsal bulge, and endosperm reflectivity; the dynamic features include angular velocity and flipping frequency.

[0015] The present invention also adopts the following technical solution: a wheat variety identification system based on a linear array camera and near-infrared spectroscopy, realizing the above-mentioned wheat variety identification method based on a linear array camera and near-infrared spectroscopy, comprising: a pulsed light source providing a high-precision light source, an edge computing device performing real-time data processing and analysis, a high-frequency linear array camera acquiring morphological information of wheat grains, and a fiber optic probe spectrometer acquiring near-infrared spectral information of wheat grains; the pulsed light source is synchronized with the edge computing device, the high-frequency linear array camera, and the fiber optic probe spectrometer through a photoelectric trigger.

[0016] The beneficial effects of this invention are: 1) High speed and high throughput: The high-frequency linear array camera and the fiber optic spectrometer are synchronized in the same window in microseconds. The wheat is collected instantly as it falls freely. It can be measured as it passes through the continuous flow without stopping or mechanical sorting. 2) Precise identification: Images and spectra are deeply fused through cross-attention, and differences in external morphology and internal composition are utilized simultaneously to maintain high differentiation among closely related and similar-looking varieties; 3) Real-time decision-making: On-site reasoning at the edge, outputting variety and quality results immediately on-site, eliminating reliance on remote servers, and supporting real-time pricing and distribution in the acquisition, warehousing, and processing stages; 4) Economical and durable: It has no rotating platform or displacement mechanism, has a compact structure and low power consumption, and can operate stably for a long time in industrial sites with dust, vibration and drastic temperature changes. It can also be quickly transferred to other grains, realizing multiple uses in one machine. Attached Figure Description

[0017] Figure 1 This is a flowchart of a wheat variety identification method based on a linear array camera and near-infrared spectroscopy according to the present invention.

[0018] Figure 2 This is a schematic diagram of the structure of a wheat variety identification system based on a linear array camera and near-infrared radiation according to the present invention.

[0019] Figure labels: 1. Pulse light source; 2. Phototrigger; 3. Fiber optic probe spectrometer; 4. High-frequency linear array camera; 5. Edge computing device; 6. Detection window; 7. Wheat grain; 8. Pipeline. Detailed Implementation

[0020] Example 1 This embodiment provides a method for wheat variety identification based on a linear array camera and near-infrared spectroscopy, referencing... Figure 1 This includes the following steps.

[0021] Step S1: Data synchronous collection.

[0022] Data synchronization is the core of this invention, ensuring that the system can efficiently and accurately collect data from different sensors simultaneously, thereby providing complete input for subsequent data processing, feature extraction and variety identification.

[0023] The photoelectric trigger start-up system precisely controls the acquisition process of the pulse light source, fiber optic probe spectrometer, and high-frequency linear array camera, enabling the three to work synchronously at the same time and acquire spectral and image data related to the sample respectively.

[0024] The specific steps for data synchronization and collection include the following aspects.

[0025] First, the functions of the pulse light source and the photoelectric trigger will be explained in detail.

[0026] Pulsed light sources are used to generate precise light pulses that illuminate the surface of wheat grains.

[0027] By using a pulsed light source to provide timed illumination, the system can collect reflectance spectra at very short time intervals, reducing the impact of external factors such as changes in ambient light on the measurement results.

[0028] Pulsed light sources typically feature high brightness and precise timing control, which can effectively enhance the signal intensity acquired by the spectrometer and improve the accuracy of the data.

[0029] The function of a photoelectric trigger is to control the synchronous operation of various components within the system, including but not limited to pulse light sources, spectrometers, and line scan cameras.

[0030] By controlling the timing relationship between various devices, it is ensured that all devices collect data at the same time.

[0031] Specifically, the photoelectric trigger activates the pulsed light source to emit light via a synchronization signal, simultaneously triggering the fiber optic probe spectrometer and the high-frequency linear array camera to begin their respective acquisition tasks, ensuring that their acquisition time windows are completely consistent and avoiding data misalignment or confusion caused by timing asynchrony.

[0032] The data acquisition process of the fiber optic probe spectrometer will be described in detail below.

[0033] Fiber optic probe spectrometers illuminate the surface of wheat grains with a near-infrared light source and receive the spectral signals reflected back from the grains. Different wheat varieties and samples may have different reflectance spectra. These spectral signals contain information about the internal chemical composition of the wheat grains, such as proteins, starches, lipids, and ash.

[0034] Near-infrared light signals reflected from wheat samples collected by a fiber optic probe are transmitted via fiber optic cable to a spectrometer for analysis. Photoelectric sensors within the spectrometer convert these light signals into digital data, representing the reflection intensity at different wavelengths. This data reflects the internal chemical composition of the wheat grains and provides effective information for variety differentiation.

[0035] The data acquisition process of the high-frequency linear array camera will be described in detail below.

[0036] A high-frequency linear array camera is responsible for capturing image information of wheat grains, working synchronously with a fiber optic probe spectrometer. The high-frequency linear array camera can quickly scan the entire wheat sample's external morphology, capturing detailed image features such as the wheat's shape, color, and texture. This morphological information is particularly important for identifying different wheat varieties, especially when varietal morphological differences are small; the image information can help identify subtle differences in appearance.

[0037] High-frequency line scan cameras can provide high-resolution image data, which is crucial for subsequent feature extraction. One advantage of line scan cameras is their ability to continuously acquire high-quality images in a short period of time, and to rapidly capture the external morphology of wheat grains. For fast-moving wheat samples, line scan cameras offer higher acquisition efficiency and accuracy.

[0038] The significance of synchronous data acquisition will be explained below.

[0039] First, there's data consistency. By controlling the photoelectric trigger, the fiber optic probe spectrometer and the high-frequency linear array camera begin data acquisition at the same time. This ensures a one-to-one correspondence between the spectral data and image data of each sample, avoiding data misalignment issues caused by acquisition delays or equipment inconsistencies. Thus, during subsequent feature extraction and model training, the system can match data acquired within the same time period, providing high-quality, reliable input data.

[0040] Secondly, there is the advantage of high efficiency. Synchronous acquisition not only improves data accuracy but also significantly enhances system efficiency. Because spectral and image data are captured within the same time window, subsequent data processing can be performed on the same dataset, reducing the time overhead of later data alignment. The system can perform real-time detection and variety identification of large batches of wheat samples in a very short time, greatly improving the automation level and efficiency of the production line.

[0041] Finally, there's the issue of accuracy. The advantage of simultaneous data acquisition lies in its ability to capture both the external morphological features and internal chemical composition information of wheat, providing richer information for subsequent feature extraction, fusion, and variety classification. Relying solely on images or spectral data can lead to low identification accuracy, especially when different varieties have very similar morphologies. The fusion of these two types of data can compensate for the shortcomings of a single data source, improving the accuracy of variety identification.

[0042] The acquired image and spectral data will be transmitted to edge computing devices for real-time processing via a high-speed data bus. Edge computing devices can process this data instantly, reducing transmission latency and ensuring real-time feedback and immediate decision-making. With the help of edge computing technology, the system can quickly classify wheat varieties on-site without relying on remote servers for data analysis, thereby improving processing speed and reducing costs.

[0043] Edge computing devices perform data preprocessing, feature extraction, and fusion based on synchronously acquired image and spectral data. Because the data is synchronized during acquisition, the data will be highly consistent in subsequent processing, providing high-quality input to the variety identification model and ensuring accurate and reliable final identification results.

[0044] Step S2: Spectral data preprocessing and feature extraction.

[0045] In wheat variety identification systems, the quality of spectral data directly affects the accuracy of subsequent analyses; therefore, preprocessing near-infrared spectral data is an essential step. The main purpose of preprocessing is to remove noise from the data, eliminate interference from the external environment, and normalize the data, thereby ensuring that the spectral data in the analysis process is more accurate and reliable.

[0046] The following section elaborates on the implementation of spectral data preprocessing, which can be performed using the standardized difference method, Savitzky-Golay (SG) smoothing method, or detrend method.

[0047] Standardized Normal Variation (SNV) is a commonly used method to remove background interference and eliminate the influence of environmental changes. By standardizing the spectral signal, the mean value of each wavelength is subtracted from the spectral data at each wavelength, and then divided by the standard deviation of that wavelength. In this way, the standardized data is unaffected by factors such as changes in light source and instrument drift, and more accurately reflects the spectral characteristics of the sample.

[0048] The standardized difference method can eliminate the scattering effect of different samples or instruments, reduce the influence of environmental factors (such as temperature and humidity) on the spectrum, make the spectral data more stable, and is suitable for batch processing of large-scale samples.

[0049] Specifically, the mean and standard deviation of the entire sample set are calculated for each wavelength point; then the mean of that wavelength point is subtracted from the sample value of each wavelength point, and divided by the standard deviation; thus, the standardized spectral data is obtained.

[0050] Savitzky-Golay smoothing is a commonly used smoothing method to eliminate high-frequency noise in spectral data. This method smooths the data by fitting a polynomial (usually a quadratic or cubic polynomial) to each data point, thus preserving the original data characteristics while reducing noise interference.

[0051] The Savitzky-Golay smoothing method can smooth high-frequency noise in data, preserve the overall trend of spectral data, and is suitable for removing random noise caused by measurement instrument errors or external environmental fluctuations.

[0052] Specifically, a fixed window is selected around each data point; polynomial fitting, such as quadratic or cubic fitting, is performed on the data within the window; the fitting result is used to replace the data points within the window to obtain smoothed spectral data.

[0053] Detrending is primarily used to remove long-term drift in spectral data. During spectral acquisition, instrument drift or environmental changes may cause the spectral signal to exhibit a gradual trend. Detrending methods eliminate these long-term variations, ensuring data stability and making the spectral data more suitable for analysis.

[0054] Detrending methods eliminate long-term trends or drift in spectral data, enhance the stability and accuracy of spectral data, and are suitable for situations where there are obvious trends or drifts in spectral signals.

[0055] Specifically, the trend of the spectral data is calculated, for example, by using a linear regression model to obtain the linear trend of the data; then the trend is subtracted from the original spectral data to obtain the detrended data.

[0056] In this embodiment, the three methods—standardized difference method, Savitzky-Golay (SG) smoothing method, and detrend method—are set as three different spectral data preprocessing modules and integrated into one unit, with the preprocessing method selected according to the sample characteristics.

[0057] After spectral data preprocessing, the next step is to extract meaningful features from the spectra. These features can help us understand the differences in chemical composition among wheat varieties. To extract effective features, this embodiment uses the Transformer model, a deep learning method based on a self-attention mechanism, which can capture key component features in spectral data.

[0058] The Transformer model was originally designed for processing sequential data (such as natural language processing tasks), and it can extract key information from the input sequence through a self-attention mechanism. In the system of this invention, the Transformer model is applied to spectral data, leveraging its powerful feature learning capabilities to extract chemical composition features related to wheat varieties. The self-attention mechanism can automatically focus on important wavelength regions in the spectrum, ignoring irrelevant information, thereby helping the model accurately identify the characteristics of wheat varieties.

[0059] The Transformer model can extract important component information (such as protein, starch, lipid, ash, etc.) from spectral data; it can focus on useful wavelength ranges and ignore unimportant signals through a self-attention mechanism; and it can enhance the ability to identify differences in chemical composition among different wheat varieties.

[0060] Specifically, the preprocessed spectral data is input into the Transformer model; the Transformer model analyzes the relationship between each wavelength point through a self-attention mechanism to identify key spectral features that affect wheat variety classification; the model outputs these features and provides high-quality input data for subsequent variety identification models.

[0061] By using the feature extraction of the Transformer model, we can extract spectral region features from spectral data. The Transformer model can identify key wavelength regions that affect variety classification. These regions often reflect the differences in chemical composition of wheat varieties. By analyzing the changes in spectral data, we can extract the differences in chemical composition (such as protein content, starch content, etc.) of wheat varieties, and further enhance the accuracy of identification.

[0062] After feature extraction, the spectral features are used as input to the subsequent variety classification model. Because spectral features can accurately reflect the chemical composition of wheat grains, combined with image features (such as morphology and color), the system can efficiently and accurately classify and identify wheat varieties.

[0063] Step S3: Image data processing and feature extraction.

[0064] Image data processing and feature extraction are crucial components of wheat variety quality identification systems. Image data provides detailed information on wheat grain morphology and color, which is essential for distinguishing different wheat varieties and assessing grain quality indicators (such as morphology, color, and damage). In fact, the chemical composition characteristics of different wheat varieties, such as protein content, damaged starch, moisture content, and ash content, are also important criteria for differentiation. These quality indicators directly affect the nutritional value, processing suitability, and market quality assessment of wheat; therefore, accurately extracting information from these quality indicators is also crucial for variety quality identification.

[0065] This invention analyzes image data from multiple dimensions, effectively extracting morphological, color, local, and dynamic features of wheat to distinguish wheat varieties. Furthermore, it integrates these features with the content of chemical components such as protein, damaged starch, moisture, and ash in spectral data, thereby improving the accuracy of wheat variety identification and quality assessment.

[0066] Wheat grain variety feature extraction includes grain morphology feature extraction, color feature extraction, local feature extraction, and dynamic feature extraction.

[0067] Morphological characteristics are analyzed based on the geometric shape of wheat grains, typically including aspect ratio, area, roundness, and symmetry. These characteristics reveal differences in the appearance of wheat grains and help distinguish between different varieties, especially when there are significant morphological differences between varieties. In wheat variety identification, ENVI software is used to extract the geometric characteristics (aspect ratio, roundness, symmetry, etc.) of wheat grains. Specifically, aspect ratio: calculated by measuring the longest and shortest diameters of the wheat grain, is particularly important for distinguishing between long and short wheat varieties; roundness: a standardized measure of wheat grain shape, measured by calculating the degree of roundness of the grain edges; symmetry: an important characteristic reflecting the regularity of its shape, with significant differences in symmetry between different wheat varieties, making symmetry an important reference indicator for variety identification.

[0068] Color is an important indicator for evaluating the appearance quality of wheat. Different wheat varieties typically exhibit different colors, and this color difference is usually related to factors such as the variety's genetic characteristics, maturity, and cultivation environment. By analyzing the color characteristics of wheat grains, different varieties can be effectively distinguished, and a preliminary judgment can be made on the processing suitability of wheat. Color feature extraction uses an RGB three-channel color histogram. By calculating the RGB three-channel color (red, green, blue) histogram of the wheat image, the color distribution information of wheat grains is obtained. Different wheat varieties usually have significant differences in color; for example, the color of wheat grains may range from light yellow to deep gold, and even have red or green hues. By using color histograms to capture these subtle color differences, the variety can be classified. By calculating and analyzing color histograms, the effects of wheat grain maturity, light exposure, and other environmental factors can be effectively assessed, helping us to further determine the variety classification criteria.

[0069] Local features refer to the small-scale geometric features of a specific region (such as the abdomen or top) of a wheat grain. Local feature extraction helps distinguish varieties with similar shapes and colors, especially when morphological differences are minor; detailed analysis of local features can provide more discriminative information. Local feature extraction involves analyzing the region of interest (ROI) of the wheat grain to extract its geometric and topological features. The abdomen region often shows subtle differences between varieties, especially when there is minor damage or irregularity in the grain surface. Geometric features are extracted from the selected region, including local edge features, regional symmetry, and angular information. These local features are particularly important for distinguishing subtle differences between varieties, especially when different varieties exhibit different details on the grain surface. Feature analysis of local regions such as the abdomen region can effectively improve the accuracy of identification between morphologically similar varieties. This method is particularly suitable for varieties with small morphological differences, such as certain similar types of hard and soft wheat varieties.

[0070] Dynamic features refer to the morphological and color changes of wheat grains during movement. These features can reflect subtle differences in varieties under different conditions, especially when wheat is in a dynamic process such as falling or rolling. Dynamic features can provide additional information about the wheat milling process, which is crucial for guiding actual production in flour mills. Dynamic feature extraction uses time-series analysis to capture changes in color, morphology, and other characteristics of wheat during movement. These changes are usually closely related to the inherent properties of the variety (such as grain hardness and surface smoothness). By recording image data of wheat grains at different time points, their morphological changes during movement are analyzed. For example, hard wheat grains and soft wheat grains may exhibit different deformation or vibration patterns during milling.

[0071] The quality characteristics of wheat grains were extracted, including protein content, damaged starch content, moisture content, and ash content.

[0072] Protein content is one of the important indicators for measuring the quality of wheat grains, and it usually affects the quality of wheat milling, processing performance and nutritional value.

[0073] Protein absorption peaks typically appear in the near-infrared spectrum, around 1700 nm to 2200 nm. By analyzing these absorption peaks, the system can accurately extract the protein content.

[0074] Therefore, near-infrared spectroscopy can be used to accurately measure the protein content in wheat grains.

[0075] In addition, protein content can be indirectly inferred from image data, because wheat kernels with higher protein content usually have a smoother surface and a more uniform color.

[0076] Morphological features extracted from image data (such as the smoothness, shape, and size of grains) are closely related to protein content.

[0077] Wheat with a high protein content often exhibits a uniform color and a smoother surface, lacking damage or irregular shapes.

[0078] Combining protein-related absorption features in the spectrum with morphological features in the image can effectively improve the accuracy of protein content prediction and enhance the classification ability of varieties.

[0079] Damaged starch is another important indicator of wheat flour quality, as it directly affects the processing performance of flour and the quality of the final product.

[0080] Damaged starch exhibits unique absorption characteristics in the spectrum, especially in the wavelength range of 1600 nm to 1800 nm.

[0081] Therefore, near-infrared spectroscopy can also be used to accurately measure the content of damaged starch in wheat grains and wheat flour.

[0082] In addition, wheat with a high content of damaged starch usually has more grain cracks and damaged surfaces, so morphological characteristics (such as edge clarity and smoothness) can reflect the content of damaged starch.

[0083] Analysis of the color histogram suggests that higher levels of damaged starch may lead to uneven coloring on the kernel surface.

[0084] By combining the absorption characteristics of damaged starch in spectral data with the irregular morphology in image features, the damaged starch content of wheat can be predicted more accurately, thereby improving the ability to identify the quality of varieties.

[0085] Moisture content is an important indicator affecting the shelf life and processing performance of wheat. Wheat with higher moisture content is usually more moist and has significant moisture absorption characteristics near 1440 nm and 1940 nm in the near-infrared spectrum.

[0086] Therefore, by analyzing these characteristics using near-infrared spectroscopy, the system can accurately estimate the moisture content in wheat grains.

[0087] In addition, wheat grains with higher moisture content usually exhibit a darker color and surface luster.

[0088] Color features in an image (such as RGB values) can be used to indirectly estimate moisture content.

[0089] By fusing the spectral characteristics of moisture content with the color features in the image, the moisture level of wheat can be identified more accurately, especially when the moisture difference between varieties is small.

[0090] Ash content is an indicator of the impurities and mineral content in wheat. Higher ash content usually indicates poorer wheat quality and may affect the nutritional composition and processing quality of flour.

[0091] The effect of ash content on the spectrum is relatively indirect, but certain wavelengths (such as around 2000 nm) may show certain characteristics that reflect the mineral content in wheat.

[0092] Meanwhile, wheat grains with higher ash content may appear rougher and more uneven on the surface. By extracting surface roughness or texture features from image data, the ash content can be indirectly estimated.

[0093] By combining spectral and image data, the ash content of wheat can be assessed more comprehensively.

[0094] Especially when the ash content difference between wheat varieties is small, image features can effectively supplement the deficiencies of spectral data and improve prediction accuracy.

[0095] Step S4: Multimodal fusion of image features and spectral data.

[0096] By fusing image data (morphology, color, local and dynamic features) with spectral data (protein, damaged starch, moisture, ash content) in a multimodal manner, the system can more comprehensively identify the quality of wheat varieties.

[0097] The fusion method can employ a cross-attention mechanism, which can effectively combine complementary information from image and spectral data to improve the accuracy and robustness of wheat variety quality identification.

[0098] By employing a cross-attention mechanism, the different importance of image features and spectral data is automatically weighted, enabling the effective fusion of features from different modalities and improving the classification accuracy and quality prediction capabilities of the final model.

[0099] The specific integration process includes the following sub-steps.

[0100] Step S41, Data Input and Preprocessing.

[0101] Image data was acquired by a high-frequency linear scan camera, providing morphological, color, and dynamic characteristics of wheat appearance. First, the image quality was improved through contrast enhancement and noise reduction; second, RGB values ​​were extracted, and a color histogram was calculated to obtain the color characteristics of wheat grains; simultaneously, the geometric features of wheat (such as aspect ratio, roundness, and symmetry) were extracted.

[0102] The contents of chemical components such as protein, damaged starch, moisture, and ash in wheat grains and during the milling process were collected using a near-infrared spectrometer. Noise was removed using SNV (Standardized Difference Method), and systematic errors caused by instrument drift and environmental factors were removed using SG (Savitzky-Golay Smoothing Method) and DETREND (Detrending Method). Then, the chemical component characteristics of wheat grains and during the milling process were extracted using feature extraction methods (such as absorption peak-based analysis).

[0103] Step S42, Cross-Attention Mechanism and Model Construction.

[0104] First, the image features and spectral data are encoded separately. The image features are encoded using a convolutional neural network (CNN), while the spectral data is encoded using a specific encoder (such as a Transformer encoder).

[0105] Next, using Cross-Attention, image features are used as keys to provide appearance information, and features in spectral data are used as queries to provide chemical composition information. The similarity between spectral features (queries) and image features (keys) is calculated.

[0106] Then, based on the calculated attention weights, the image features (values) are weighted and fused into the spectral features. In this way, each image feature is weighted according to its similarity to the spectral features, thus forming a new, fused feature value.

[0107] The weighted fusion values ​​obtained through the cross-attention mechanism are input into wheat variety quality identification models (such as machine learning models or deep learning models such as support vector machines (SVM), gradient boosting decision trees (GBDT), and convolutional neural networks (CNN)) to classify and evaluate wheat varieties.

[0108] At this point, the system can not only efficiently distinguish different varieties of wheat, but also predict their quality indicators (such as protein, damaged starch, moisture, ash content, etc.).

[0109] Step S43, Model Training and Evaluation.

[0110] Using wheat sample data from the training set, the model learns the relationship between image features and spectral data, and how they work together to achieve accurate variety classification and quality assessment.

[0111] The training process typically includes techniques such as backpropagation and optimizer tuning (e.g., the Adam optimizer), which enable the model to continuously adjust its parameters and improve prediction accuracy.

[0112] The trained model was evaluated using a test set. The main evaluation metrics included: 1) classification accuracy, which evaluates the accuracy of wheat variety classification; 2) mean squared error (MSE), which evaluates the prediction error of quality indicators (such as protein, moisture, etc.); and 3) F1 score, precision, and recall, which comprehensively evaluate the model's performance in different categories.

[0113] By training and testing on a large number of wheat samples, the system of this invention can efficiently fuse spectral data with image data and accurately classify wheat varieties. Through a cross-attention mechanism, the system can extract the most useful information from the spectral and image data, improve the classification accuracy of wheat varieties, and predict key quality indicators of wheat, such as protein content, damaged starch, moisture content, and ash content.

[0114] This system can be applied to: 1) agricultural production lines, automating variety classification and quality testing to improve production efficiency; 2) breeding and screening, helping breeding experts to screen wheat varieties with excellent quality; 3) quality control, monitoring wheat quality in real time to ensure product standardization and consistency.

[0115] Step S5, edge reasoning.

[0116] The trained and validated wheat variety quality identification model will be optimized and transformed using the TensorRT engine to adapt to edge computing devices.

[0117] With acceleration and optimization through TensorRT, the model can significantly improve inference speed when running on edge devices, ensuring real-time performance.

[0118] Based on this, the model will be deployed to edge computing devices (such as Jetson Nano) and integrated with the actual detection process.

[0119] Once deployed, the system can perform real-time wheat variety and quality identification on-site without relying on a remote server, enabling immediate on-site feedback and rapid decision-making.

[0120] Edge computing devices can process large amounts of real-time data, ensuring stable operation of the system in high-throughput environments such as production lines.

[0121] Example 2 This embodiment provides a wheat variety identification system based on the fusion of a high-frequency linear array camera and near-infrared fusion. (Reference) Figure 2 This system integrates optical imaging technology and near-infrared spectroscopy analysis technology, and through the collaborative work of a high-frequency linear array camera, a fiber optic probe spectrometer, and edge computing devices, it can identify wheat varieties and assess their quality in real time and efficiently.

[0122] The system includes a pulsed light source 1, a photoelectric trigger 2, an edge computing device 5, a high-frequency linear array camera 4, and a fiber optic probe spectrometer 3.

[0123] Pulsed light sources can provide a stable light source, ensuring that wheat grains receive uniform light during harvesting.

[0124] The photoelectric trigger controls the synchronous operation of the light source, camera, and spectrometer to ensure precise matching of the time points for each data acquisition.

[0125] Edge computing devices are connected to high-frequency linear array cameras and fiber optic probe spectrometers to process data in real time, perform feature extraction, data fusion, and model inference, reduce data transmission latency, and enable rapid on-site decision-making.

[0126] High-frequency linear array cameras can capture the morphological, color, local, and dynamic characteristics of wheat grains in a short time, and identify the varieties of wheat grains moving at high speeds in real time.

[0127] Fiber optic probe spectrometers collect near-infrared spectral data of wheat grains to obtain the characteristics of the internal chemical composition of wheat, such as protein, starch, lipids, and ash.

[0128] To achieve efficient wheat variety identification, this system uses a transparent acrylic tube as the free-fall channel for wheat grains. The inner wall of the tube undergoes a special low-friction treatment to ensure that the coefficient of friction does not exceed 0.05, thereby reducing friction between the grains and the tube and ensuring that the wheat grains can fall freely without external interference.

[0129] Specifically, in this embodiment, the system includes a pipe 8, a detection window 6, a high-frequency linear array camera, a pulsed LED light source, a fiber optic probe spectrometer, and a precision triggering device.

[0130] The use of transparent acrylic tubing ensures clear observation of wheat grains while avoiding optical distortion caused by uneven material distribution. The low-friction treatment of the tubing's inner wall ensures that the wheat grains fall at a consistent speed.

[0131] A detection window is set in the middle section of the pipeline. The window position is optimized according to the configuration of the fiber optic probe and the high-frequency linear array camera to ensure that wheat grains can pass through and have their spectral data and image data collected simultaneously.

[0132] Fiber optic probe spectrometers obtain the chemical composition characteristics of wheat by directly contacting the sample with the fiber optic probe.

[0133] The line frequency requirement of the high-frequency linear array camera is no less than 8000 Hz to ensure that image data can be captured in real time during the rapid fall of wheat grains, thus guaranteeing image acquisition accuracy.

[0134] The light source uses a 4500K high color rendering LED with a pulse time as short as 10 microseconds to ensure rapid illumination and provide clear image data.

[0135] This spectrometer covers a wavelength range of 900-1700 nanometers and can accurately acquire near-infrared spectral data of wheat grains. The spectrometer and camera acquire data simultaneously at the same detection window position.

[0136] A photoelectric trigger with a response time of 1 microsecond is connected to the aforementioned device. When wheat grains pass through the detection window, the triggering device simultaneously activates the pulsed LED light source, the high-frequency linear array camera, and the fiber optic probe spectrometer to collect spectral data, ensuring that all data are acquired synchronously.

[0137] With the above equipment configuration, image features and spectral data of each wheat grain are collected synchronously.

[0138] For image features, a high-frequency linear array camera acquires image information of wheat grains, and the image data is saved in JPG format. The coordinates of each image are labeled to ensure that each image corresponds one-to-one with the corresponding spectral data and to ensure the correlation of the data.

[0139] For spectral data, a fiber optic probe spectrometer was used to acquire near-infrared spectral information of wheat grains. The spectral data for each grain was saved as a numerical matrix, containing the reflectance intensity corresponding to multiple wavelength points. The data was saved in CSV format for easy subsequent analysis and processing.

[0140] The collected image data is processed and features are extracted to extract the appearance features of wheat.

[0141] The appearance characteristics of wheat include morphological characteristics, color characteristics, local characteristics, and dynamic characteristics.

[0142] The morphological characteristics of wheat were extracted using ENVI software, including aspect ratio, rectangularity, and surface area to volume ratio.

[0143] For aspect ratio, the shape characteristics of wheat grains are evaluated by measuring the ratio of the main axis length to the maximum width.

[0144] For rectangularity, the regularity of the grain is assessed by the ratio of the grain area to the area of ​​the smallest bounding rectangle.

[0145] The surface area to volume ratio is used to assess the surface smoothness of wheat grains by calculating the ratio of apparent area to projected volume.

[0146] By extracting RGB three-channel feature values ​​from color histograms, the color distribution characteristics of wheat can be obtained, reflecting the maturity and varietal differences of wheat.

[0147] The ventral region of wheat grains was selected for ROI (Region of Interest) extraction. The embryonic end concavity (reflecting the degree of embryonic end concavity by the eccentricity of the fitted ellipse), ventral curvature (integrating the curvature along the principal axis to assess the degree of curvature of the wheat grain's abdomen), dorsal bulge (calculating the distance between the highest point of the grain's dorsal side and the fitted plane to assess the degree of dorsal bulge), and endosperm reflectivity (calculating the mean gray value of the ROI region to reflect the reflectivity of the wheat grain's endosperm) were extracted to obtain local features.

[0148] The dynamic characteristics of wheat grains are extracted by time series analysis to identify the angular velocity and tumbling frequency of wheat grains during the falling process, reflecting the motion state of the grains.

[0149] Then, the collected spectral data were preprocessed using SNV (Standardized Interference), SG (Savitzky-Golay Smoothing), and DETREND (Detrending) methods to eliminate noise and improve data quality.

[0150] By using the Transformer model to extract wavelength-related features from the raw spectral data in the range of 1650 nm to 1700 nm, and combining this with chemometrics, protein content can be accurately assessed. Characteristic bands of damaged starch in the range of 1580 nm to 1670 nm are automatically identified, and the content of damaged starch is determined based on changes in these peaks. Moisture features in the bands around 1400 nm and 1900 nm are extracted to capture the correlation between moisture content and the spectrum, accurately predicting moisture content. Ash content characteristic bands around 2000 nm are extracted to capture changes in mineral absorption bands, thereby predicting ash content in wheat.

[0151] Next, using Cross-Attention, image features are used as keys and spectral data features as queries. Attention weights are calculated to weight the correlation between modalities. In this way, the model can optimize the feature fusion process based on the correlation between spectral and image data, thereby improving the accuracy of wheat variety classification.

[0152] Next, we will establish a wheat variety quality identification model.

[0153] Based on the feature fusion results, a wheat variety quality identification model was established using the SNV (Standardized Interference Method) and GBD (Gradient Boosting Decision Tree) algorithms. First, the fused features were input into the GBD model for training. The GBD algorithm, by integrating multiple decision trees, can effectively handle complex nonlinear relationships and improve the model's predictive performance. Then, cross-validation and parameter tuning were used to optimize the GBD model's performance, ensuring its accuracy in identifying different varieties.

[0154] Finally, model deployment and edge computing are carried out.

[0155] The trained identification model is converted into the TensorRT engine and deployed on a Jetson Nano edge computing device. This enables the system to identify wheat variety quality in real-time on-site, ensuring rapid feedback and efficient processing.

[0156] TensorRT optimizes trained models, enabling them to run efficiently on edge devices, reducing latency and increasing processing speed.

[0157] The optimized TensorRT engine was deployed to Jetson Nano, and real-time data was analyzed through edge computing devices to complete the automated identification and quality assessment of wheat varieties.

[0158] This invention is also applicable to the identification of varieties of grains such as barley, oats, and rice, and its technical solution covers any seed / grain classification device that utilizes image and near-infrared fusion.

[0159] This invention features high efficiency, accuracy, and real-time performance, significantly improving the screening efficiency of wheat varieties in agriculture. By combining high-precision morphological and chemical composition analysis, it overcomes the limitations of traditional identification methods, demonstrating significant application prospects, particularly suitable for automated detection and variety control in modern agricultural production lines.

[0160] First, high efficiency and high precision.

[0161] This invention combines a high-frequency linear array camera with a near-infrared spectrometer to simultaneously acquire morphological and chemical composition information of wheat grains, thereby efficiently identifying wheat varieties and quality in a short time. Compared to traditional identification methods, this method (system) not only improves the identification speed but also ensures higher identification accuracy, especially for wheat varieties with similar morphology but different internal components, demonstrating significant advantages.

[0162] Second, real-time detection and field application.

[0163] The introduction of edge computing devices enables the entire system to process data in real time, avoiding the real-time issues caused by data transmission delays in traditional methods. The system can process wheat varieties in real time the instant they are collected, making it suitable for modern industrial production lines. It can quickly screen out qualified or unqualified wheat varieties, meeting the needs of large-scale, high-throughput testing.

[0164] Third, high integration and cost-effectiveness.

[0165] This invention integrates a high-frequency linear array camera with a fiber optic probe spectrometer, forming a highly integrated system that reduces equipment redundancy and improves overall system efficiency. The system hardware structure is relatively simple, easy to operate, and can achieve efficient variety identification at a low cost, showing promising market application prospects.

[0166] Fourth, it is highly adaptable.

[0167] This invention's system can process wheat samples under various environmental conditions. Whether in a humid or dry environment, the system operates stably, acquiring accurate spectral and image information. Furthermore, this system can not only be used for wheat variety identification but also, by adjusting relevant parameters, can be extended to the quality control of other crops, demonstrating strong versatility.

Claims

1. A method for wheat variety identification based on a linear array camera and near-infrared spectroscopy, characterized in that, include: S1, within the same detection window along the free fall path of the grain, acquire image data and near-infrared spectral data of the same grain respectively; S2, the spectral data is preprocessed within the edge computing device, and chemical composition features are extracted using the Transformer self-attention mechanism; S3, preprocess the image data to extract the image features of the seeds; S4 uses a cross-attention mechanism to use image features as keys and spectral features as queries, calculates similarity and fuses them to obtain a fused feature vector. S5 inputs the fused feature vectors into a lightweight model optimized by TensorRT, completes variety classification and quality index prediction on an edge computing device, and outputs the results in real time.

2. The method for wheat variety identification based on a linear array camera and near-infrared spectroscopy according to claim 1, characterized in that, In step S4, the cross-attention mechanism includes: spectral features are encoded by Transformer as queries, image features are encoded by CNN as keys, weights are calculated by scaling dot product attention, and appearance information is weighted and injected into chemical features to form a fused feature vector.

3. The method for wheat variety identification based on a linear array camera and near-infrared spectroscopy according to claim 1, characterized in that, In step S5, the TensorRT optimized model compresses the original GBDT or CNN model to less than 100 MB through knowledge distillation, and the single-seed inference time on the Jetson Nano edge device does not exceed 20 ms.

4. The method for wheat variety identification based on a linear array camera and near-infrared spectroscopy according to claim 1, characterized in that, In step S1, obtaining image data and near-infrared spectral data of the same grain includes: a microsecond-level synchronization signal is emitted by a photoelectric trigger to trigger the pulse light source, high-frequency linear array camera and fiber optic probe-type near-infrared spectrometer to work simultaneously to obtain image data and near-infrared spectral data of the same grain.

5. The method for wheat variety identification based on a linear array camera and near-infrared spectroscopy according to claim 1, characterized in that, In step S2, the spectral data preprocessing is performed using the standardized difference method, Savitzky-Golay smoothing method, or detrending processing.

6. The method for wheat variety identification based on a linear array camera and near-infrared spectroscopy according to claim 1, characterized in that, In step S3, image data preprocessing includes denoising, enhancement, binarization, and contour extraction.

7. The method for wheat variety identification based on a linear array camera and near-infrared spectroscopy according to claim 2, characterized in that, The line frequency of the high-frequency linear array camera is not less than 8000 Hz, and the exposure time is not more than 10 μs.

8. The method for wheat variety identification based on a linear array camera and near-infrared spectroscopy according to claim 1, characterized in that, In step S4, the fused feature vector is obtained through intermediate feature fusion.

9. A method for wheat variety identification based on a linear array camera and near-infrared spectroscopy according to claim 1 or 2, characterized in that, The image features include morphological features, color features, local features, and dynamic features; the morphological features include rectangularity and surface area to volume ratio; the color features are obtained through RGB three channels; the local features include embryonic end depression, abdominal curvature, dorsal bulge, and endosperm reflectivity; the dynamic features include angular velocity and flipping frequency.

10. A wheat variety identification system based on a line scan camera and near-infrared spectroscopy, implementing the wheat variety identification method based on a line scan camera and near-infrared spectroscopy as described in any one of claims 1-9, characterized in that, include: The pulsed light source provides a high-precision light source, the edge computing device performs real-time data processing and analysis, the high-frequency linear array camera collects the morphological information of wheat grains, and the fiber optic probe spectrometer collects the near-infrared spectral information of wheat grains; the pulsed light source is synchronized with the edge computing device, the high-frequency linear array camera, and the fiber optic probe spectrometer through a photoelectric trigger.

Images