Rice inspection method and system based on light sensing technology
By combining visible-near-infrared hyperspectral imaging and structured light projection technology, and utilizing graph convolutional neural networks and depth metric learning algorithms, high-precision detection of rice defects and varieties has been achieved, solving the problems of insufficient detection accuracy and reliability in existing technologies.
Patent Information
- Application Number
- CN202511406990.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-29
- Publication Date
- 2026-01-13
AI Technical Summary
Existing rice testing technologies cannot simultaneously achieve accurate detection of rice defects and accurate identification of varieties in complex production environments, resulting in insufficient detection accuracy and reliability.
By employing visible-near-infrared hyperspectral imaging technology and structured light projection technology, combined with graph convolutional neural networks and depth metric learning algorithms, three-dimensional point cloud data is acquired through multimodal data fusion and adaptive adjustment of structured light projection parameters, and defect identification and variety classification are performed.
It significantly improved the accuracy of rice defect detection by 15% and variety identification by 12%, solved the spatial registration problem of multimodal data, and ensured the accuracy of variety identification in the presence of defects.
Smart Images
Figure CN121330664A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of light sensing, more particularly, it relates to a rice inspection method and system based on light sensing technology, which utilizes visible-near infrared hyperspectral imaging technology and structured light projection technology, combines graph convolutional neural network and deep metric learning algorithm, and realizes intelligent detection method of defect detection, variety identification and quality grading of rice samples. BACKGROUND
[0002] With the continuous improvement of consumers' requirements for rice quality, the rice processing industry urgently needs intelligent detection technology with high precision and high efficiency to ensure product quality. In modern intelligent rice sorting production lines, defect detection and variety identification of rice have become key technical links.
[0003] Existing rice detection technologies mainly adopt machine vision methods, including two-dimensional detection technology based on RGB images, hyperspectral imaging technology and structured light three-dimensional measurement technology. However, these single-mode detection methods have obvious limitations in practical application.
[0004] The existing technology has the following main defects: first, the traditional two-dimensional image detection method is difficult to accurately identify three-dimensional morphological defects such as insect damage and cracks, and the detection precision is limited; second, although the hyperspectral imaging technology can provide rich spectral information for variety identification, it is affected by the shadow blocking caused by three-dimensional defects, resulting in incomplete spectral data; third, the structured light three-dimensional measurement technology can obtain morphological information, but the surface reflection characteristics of different varieties of rice are different, resulting in inconsistent phase solving errors, affecting the measurement precision; fourth, the existing methods lack effective fusion mechanism of multi-modal data, and the spatial registration precision of morphological data and spectral data is insufficient.
[0005] These technical defects lead to a major technical problem: it is impossible to simultaneously realize accurate detection of rice defects and accurate identification of varieties in complex production environments, which seriously affects the detection precision and reliability of the rice intelligent sorting system. SUMMARY
[0006] The present application provides a rice inspection method based on light sensing technology, which solves the technical problem of simultaneous realization of accurate detection of rice defects and accurate identification of varieties in related technologies.
[0007] The application discloses a rice inspection method based on light sensing technology, comprising the following steps: collecting hyperspectral image data of a rice sample to be inspected by using a visible-near-infrared hyperspectral imaging system, calculating reflectivity distribution of each wave band, segmenting the image into different reflection characteristic regions based on the reflectivity distribution and texture features by using a clustering algorithm, and generating a spectral partition mask; calculating the optimal projection light intensity of each region according to the average reflectivity of each region in the spectral partition mask, projecting a multi-frequency sinusoidal fringe sequence in a region-by-region manner, obtaining absolute phase by using a multi-frequency phase unwrapping algorithm and calculating three-dimensional height distribution, generating three-dimensional point cloud data, and identifying a defect candidate region; analyzing the fringe pattern in the defect candidate region, extracting a geometric feature vector, jointly decomposing spectral data, texture features and depth information by using a non-negative matrix factorization method, and generating a multi-modal feature tensor; constructing the defect candidate region into a graph structure, inputting the multi-modal features as node features into a graph convolutional neural network, aggregating neighborhood information and updating node representation through a graph convolutional layer, respectively performing defect type classification, severity evaluation and variety origin identification, and outputting a defect-variety comprehensive evaluation matrix; wherein the calculation of the optimal projection light intensity is based on the inverse relationship between the reference light intensity and the average reflectivity of the region, the multi-frequency phase unwrapping algorithm obtains the absolute phase by solving the integer multiple through frequency combination, and the graph convolutional layer aggregates the node features by weighting through an adjacency matrix and a degree matrix.
[0008] The application discloses a rice inspection system based on light sensing technology, which is used for executing the rice inspection method based on light sensing technology, and comprises a visible-near-infrared hyperspectral imaging system, which is used for collecting hyperspectral image data of a rice sample; a structured light projection system, which is used for adaptively adjusting projection parameters according to a spectral partition mask and projecting a multi-frequency sinusoidal fringe sequence; an image processing unit, which is used for calculating reflectivity distribution, generating a spectral partition mask, and extracting multi-modal features; a graph convolutional neural network processing unit, which is used for graph structure modeling and comprehensive evaluation of the multi-modal features; and a depth metric learning unit, which is used for correcting variety identification results and generating quality grading decisions.
[0009] Further, the step of generating the spectral partition mask comprises the following steps: extracting a gray level co-occurrence matrix of each wave band image, calculating contrast, correlation, energy and homogeneity texture features, and generating a texture feature vector set; performing standardization pretreatment on the reflectivity features and the texture features, and performing mean-variance standardization processing on the feature vectors; grouping pixels by minimizing the weighted sum of square distances between the pixels and the cluster centers, outputting a cluster label set and a cluster center, and generating the spectral partition mask.
[0010] Further, the step of obtaining the three-dimensional point cloud data comprises projecting a multi-frequency sinusoidal fringe sequence containing low, medium and high frequencies according to the calculated optimal projection light intensity; calculating the wrapped phase under each frequency by using a four-step phase shifting method, wherein the four-step phase shifting fringe images are 0-degree, 90-degree, 180-degree and 270-degree phase shift images; obtaining the absolute phase by combining the low, medium and high frequency wrapped phases through a multi-frequency phase unwrapping algorithm; and calculating the three-dimensional height distribution based on the linear mapping relationship between the absolute phase and the system calibration distance to generate the three-dimensional point cloud data.
[0011] Further, the step of generating the multi-modal feature tensor comprises calculating the defect depth according to the fringe break position and the deformation degree, wherein the product relationship between the phase jump value and the system working distance is used to determine the defect depth; extracting the geometric feature vector of the defect area, including the defect area, the perimeter, the maximum depth, the average depth and the volume; performing maximum-minimum normalization preprocessing on the geometric feature vector to normalize each feature value to the interval of zero to one; and performing unified normalization preprocessing on the multi-modal data, decomposing the original matrix under the non-negative constraint by minimizing the reconstruction error to obtain the basis matrix and the coefficient matrix.
[0012] Further, the step of processing by the graph convolutional neural network comprises constructing the defect candidate area into a graph structure, wherein the node represents a local area, and the edge is constructed based on the spatial adjacency relationship and the feature similarity; taking the geometric features of the defect area, the spectral features of the surrounding intact area and the global texture distribution as the node features; performing two-norm normalization preprocessing on the node feature vector to ensure that each type of feature participates in network calculation under a unified scale; and the output layer comprises three parallel fully connected layers, which respectively output the defect classification probability, the severity probability and the variety identification probability.
[0013] Further, a multi-task loss function is used for network training, and the multi-task loss function is a weighted combination of the defect classification cross-entropy loss, the severity classification cross-entropy loss and the variety identification cross-entropy loss.
[0014] Further, after outputting the defect-variety comprehensive evaluation matrix, the method further comprises calculating the Euclidean distance between the current sample feature and the feature of each variety in the standard variety library by using a deep metric learning model; searching for a preset spectral distortion compensation coefficient table according to the defect type and the severity to obtain a compensation coefficient; modifying the distance metric by using the compensation coefficient to recalculate the variety similarity; and comprehensively evaluating the defect evaluation result and the corrected variety identification result to generate a final quality grading decision according to a preset grading rule.
[0015] Further, the deep metric learning model is based on a multi-layer perceptron structure including an input layer, a hidden layer and an output layer, adopts a contrast learning training mode, and is trained using a triplet loss function, the triplet loss function including an anchor sample feature, a positive sample feature, a negative sample feature and a boundary parameter.
[0016] Further, the quality grading decision includes calculating a comprehensive quality score according to a defect type weight, a severity weight and a variety matching degree, wherein the sum of the defect type weight and the variety matching degree weight is one; based on a quality score threshold, the rice is divided into four grades of superior product, first-class product, second-class product and unqualified product, and specific defect description, variety identification result and suggested processing mode are output.
[0017] The present application effectively solves the problem of uneven phase calculation error caused by the difference in surface reflection characteristics of different varieties of rice by adaptively adjusting the structured light projection parameters based on the reflection characteristic distribution of different regions obtained by spectral pre-scanning, significantly improves the three-dimensional morphology reconstruction accuracy; the spectral-texture-depth three-element joint feature extraction is realized by the non-negative matrix decomposition technology, the complementarity of multi-source information is fully utilized, the problem of local information loss caused by defect shadow shielding is effectively overcome, and the technical problem of incomplete single modal data affecting detection accuracy is solved; by constructing a graph structure and using a graph convolutional neural network to model the spatial relationship, the problem of loss of spatial correspondence caused by independent processing of morphology data and spectral data is innovatively solved, and high-precision spatial registration of multi-modal data is realized; by combining the depth metric learning with the spectral distortion compensation mechanism, the interference of spectral feature shift caused by defect state change on variety identification is effectively suppressed, the accuracy of variety identification in the presence of defects is ensured, and a significant technical effect of improving the rice defect detection accuracy by 15% and the variety identification accuracy by 12% is achieved. BRIEF DESCRIPTION OF DRAWINGS
[0018] Figure 1 is the overall flowchart of the rice inspection method based on light sensing technology of the present application, including obtaining hyperspectral images to generate spectral partition masks, adaptively adjusting structured light projection parameters to obtain three-dimensional point cloud data, extracting multi-modal features to generate feature tensors, graph convolutional neural network processing to output comprehensive evaluation matrix, and depth metric learning correction variety identification result to generate quality grading decision complete process; Figure 2 is the detailed flowchart of step 1 of the present application for obtaining hyperspectral images and generating spectral partition masks, including specific steps of collecting hyperspectral image data, calculating reflectance distribution, extracting texture features, K-means clustering segmentation and spectral angle matching to generate a preliminary variety classification map; Figure 3This is a detailed flowchart of step 2 of the present invention, which is based on the adaptive adjustment of structured light projection parameters using spectral partitioning mask to obtain three-dimensional point cloud data. It includes specific steps such as calculating the optimal projection light intensity, multi-frequency fringe projection, calculating the wrapping phase using the four-step phase shifting method, obtaining the absolute phase by multi-frequency phase unfolding, calculating the three-dimensional height distribution, and identifying candidate defect regions. Figure 4 This is a detailed flowchart of step 3 of the present invention, which analyzes the stripe deformation pattern and extracts multimodal features. It includes specific steps such as ray tracing analysis to estimate the defect depth, extracting geometric feature vectors, non-negative matrix decomposition to extract ternary joint features, and fusing to generate multimodal feature tensors. Figure 5 This is a detailed flowchart of step 4 of the present invention, which includes the specific steps of constructing the graph structure, inputting node features into a multi-task network, aggregating neighborhood information in the graph convolutional layer to update node representations, and outputting a comprehensive evaluation matrix through parallel task heads. Figure 6 This is a detailed flowchart of step 5 of the present invention, which corrects the variety identification results based on deep metric learning. It includes the specific steps of calculating the Euclidean distance between the sample and the standard variety library, finding the spectral distortion compensation coefficient, correcting the distance metric to recalculate the similarity, and generating the final quality grading decision. Detailed Implementation
[0019] On high-end intelligent rice sorting production lines, traditional two-dimensional image detection methods struggle to accurately identify three-dimensional morphological defects such as insect infestation and cracks, and cannot simultaneously achieve precise identification of different origins and varieties. Existing technologies suffer from the following problems: differences in surface reflectivity among different rice varieties lead to varying errors in structured light phase calculation, affecting the accuracy of three-dimensional morphological reconstruction; shadows caused by three-dimensional defects affect the integrity of hyperspectral data acquisition, resulting in decreased variety identification accuracy; insufficient spatial registration accuracy between morphological and spectral data affects the accuracy of multi-source data joint analysis; and the spectral characteristics of the same rice variety change under different defect states, interfering with variety identification results.
[0020] The method of this embodiment includes the following steps: The rice inspection method based on photosensitive technology provided in this embodiment is performed on inspection equipment equipped with a visible-near-infrared hyperspectral imaging system and a structured light projection system. The method includes the following steps: Step 1: Obtain visible-near-infrared hyperspectral images of rice samples, extract reflectance distribution, and generate spectral partition masks.
[0021] Step 1 includes: Step 101: Use a visible-near-infrared hyperspectral imaging system to acquire hyperspectral image data of the rice sample to be tested in the 400-1000nm band, and obtain a three-dimensional data cube containing spatial and spectral dimensions.
[0022] Step 102: Calculate the reflectance distribution of each band based on the standard whiteboard correction method to obtain the reflectance data cube. ,in For spatial coordinates, λ is the wavelength.
[0023] Step 103: Extract the gray-level co-occurrence matrix of each band image, calculate the four texture features of contrast, correlation, energy and homogeneity, and generate a texture feature vector set.
[0024] Step 104: Based on reflectance distribution and texture features, the image is segmented into regions with different reflectance characteristics using the K-means clustering algorithm to generate spectral partition masks. The algorithm takes a set of feature vectors as input. ,in Total number of pixels For pixel index, Includes reflectivity features and texture features Since reflectivity features and texture features have different numerical ranges and dimensions, Z-score normalization preprocessing is required for the feature vectors. , ,in and These represent the mean and standard deviation of reflectance and texture features, respectively; the output is a set of clustered labels. and cluster center ,in The number of cluster centers. For cluster center index, For the first Clustering labels of pixels, For the first Identify cluster centers and minimize the objective function. Implement pixel grouping.
[0025] It should be noted that each region in the spectral partition mask represents a set of pixels with similar reflective properties, which are used for subsequent adaptive adjustment of the structured light projection parameters.
[0026] In this embodiment of the application, in order to improve the accuracy of preliminary variety identification, after step 104, the following step is further included: Step 105: Matching the average spectral curve of each partition with a preset variety spectral library, calculating the similarity using a spectral angle matching algorithm, and generating a preliminary variety classification map; the input of this algorithm is the spectral vector to be tested. and reference spectral vector The output is a spectral angle. Spectral similarity is determined by comparing the size of the angles.
[0027] Step 2: Adaptively adjust the structured light projection parameters based on the spectral partition mask to obtain 3D point cloud data and identify candidate defect regions.
[0028] Step 2 includes: Step 201: Based on the spectral partition mask The average reflectance of each region is used to calculate the optimal projected light intensity for each region. ,in As the reference light intensity, For the first Average reflectance of each region.
[0029] Step 202: Based on the calculated optimal projected light intensity, project a multi-frequency sinusoidal fringe sequence into different regions, with the frequency set as follows: ,in For low frequency, For medium frequency, For high frequencies, a four-step phase-shift fringe pattern is projected for each frequency.
[0030] Step 203: Calculate the wrap-around phase at each frequency using the four-step phase-shifting method. The absolute phase is obtained through a multi-frequency phase expansion algorithm. The four-step phase-shifting method inputs four phase-shifted fringe images. ,in For 0° phase shift images, For 90° phase-shifted images, For 180° phase-shifted images, The output is a 270° phase-shifted image, wrapped phase. The multi-frequency phase expansion algorithm takes multiple frequencies of wrapped phase as input. ,in For low-frequency wrapping phase, For intermediate frequency wrap phase, The high-frequency wrapping phase output is an absolute phase output. By frequency combination Solving for integer multiples .
[0031] Step 204: Based on the phase-height mapping relationship Calculate the three-dimensional height distribution, where It is an absolute phase (dimensionless). The system is calibrated with a distance constant (mm), and the height value (mm) is obtained through the linear mapping relationship between phase difference and physical height, thus generating three-dimensional point cloud data.
[0032] Step 205: Analyze the local gradient of the phase distribution and identify phase abrupt change points. The location, among which The region is marked as a candidate region for defect based on a preset threshold.
[0033] It should be noted that adaptive projection light intensity adjustment can compensate for the differences in reflectivity of different rice varieties and improve the accuracy of phase calculation.
[0034] The aforementioned adaptive projection light intensity adjustment method establishes a mapping relationship between reflectivity and optimal projection light intensity by pre-analyzing the spectral reflectance distribution of the rice surface. This overcomes the problem of inconsistent phase calculation accuracy for different rice varieties caused by traditional fixed-intensity projection; function The specific implementation is as follows: First, the reflectivity value A logarithmic transformation is performed to linearize the reflection-intensity relationship, followed by mapping using a piecewise linear function for low reflectivity regions. High gain coefficient For medium reflectivity regions Using standard gain coefficient For high reflectivity regions Use low gain coefficient Ultimately, the light intensity is limited to a reasonable range using the sigmoid function, specifically in the form of... ,in Based on light intensity, The corresponding bias parameters are used; this method dynamically adjusts the structured light projection parameters according to the reflectivity characteristics of each region, so that each region can obtain a suitable stripe contrast, thereby improving the uniformity and accuracy of the overall three-dimensional reconstruction.
[0035] Step 3: Analyze the stripe deformation pattern and extract multimodal features to generate a feature tensor.
[0036] Step 3 includes: Step 301: Perform ray tracing analysis on the stripe pattern within the defect candidate area, and estimate the defect depth based on the stripe break location and deformation degree. ,in For phase jump variables, This refers to the system's working distance.
[0037] Step 302: Extract the geometric feature vector of the defect region ,in The defect area is... For the perimeter, For maximum depth, The average depth, For volume; due to geometric features, they have different physical dimensions (area is...). Circumference is Depth is Volume is The feature vectors need to undergo maximum-minimum normalization preprocessing: Normalize each eigenvalue to Within the interval, the influence of dimensional differences on subsequent calculations is eliminated.
[0038] Step 303: Use nonnegative matrix factorization to jointly decompose the spectral data, texture features, and depth information to obtain the basis matrix. sum coefficient matrix Extracting spectral-texture-depth ternary joint features; the algorithm takes a non-negative data matrix as input. This includes spectral, texture, and depth features; since spectral data, texture features, and depth information have different numerical ranges and dimensions, unified normalization preprocessing is required for multimodal data. Ensure that each row of data in the matrix is normalized to Within the specified range, the influence of dimensional differences between different modes on matrix decomposition is eliminated; the output is the basis matrix. sum coefficient matrix By minimizing the reconstruction error Decompose the original matrix under non-negativity constraints.
[0039] Step 304: Fuse geometric features, spectral features, and texture features to generate a multimodal feature tensor. ,in For spatial dimensions, For feature dimensions.
[0040] In this embodiment of the application, in order to improve the accuracy of the internal structure analysis of the defect, step 301 further includes: based on the nonlinearity of the stripe deformation, using a quadratic polynomial to fit the stripe edge to infer the internal structure type of the defect.
[0041] The aforementioned spectral-texture-depth ternary joint feature extraction method achieves unified representation of multi-source heterogeneous data through nonnegative matrix factorization, and establishes a spectral feature matrix. Texture feature matrix and deep feature matrix Joint decomposition model , where the basis matrix Each column represents a basic pattern, the coefficient matrix This represents the weight combination of each mode, which effectively integrates complementary multimodal information and improves the robustness and accuracy of defect detection.
[0042] Step 4: Input the multimodal features into the graph convolutional neural network and output the defect-variety comprehensive evaluation matrix.
[0043] Step 4 includes: Step 401: Construct the defect candidate region into a graph structure , where nodes Represents a local area, edge It is constructed based on spatial adjacency and feature similarity.
[0044] Step 402: Input the geometric features of the defective region, the spectral features of the surrounding intact region, and the global texture distribution as node features into the multi-task graph convolutional neural network; since the geometric features, spectral features, and texture features have different numerical ranges and dimensions, L2 normalization preprocessing is required for the node feature vectors: This ensures that all features participate in network computation at a uniform scale; the network consists of an input layer, three graph convolutional layers, pooling layers, and an output layer, where the input layer receives node feature vectors. , The node feature dimension; the calculation formula for the graph convolutional layer is: ,in It is an adjacency matrix. For degree matrix, For the first The layer's weight matrix; the output layer contains three parallel fully connected layers, each outputting the defect classification probability. Severity probability and variety identification probability .
[0045] Step 403: Aggregate neighborhood information through graph convolutional layers and update node representations. ,in For nodes The neighborhood, For node degree, For the first The weight matrix of the layer, This is the activation function.
[0046] Step 404: After multi-layer graph convolution processing, three parallel task heads are used to classify defect types, assess severity, and identify variety origins, respectively, outputting a defect-variety comprehensive evaluation matrix. ,in Number of defect types For severity levels, The number of varieties; the probability vector output by the network needs to be decoded into specific classification results by Argmax: defect type is decoded as Severity decoded as Variety identification decoding At the same time, set a confidence threshold. When the maximum probability value is below a threshold, it is labeled as "uncertain". This network uses supervised learning training mode, employs the Adam optimizer for parameter optimization, and the multi-task loss function is: ; in Cross-entropy loss for defect classification, Cross-entropy loss is used to classify severity. Cross-entropy loss for variety identification, where For sample index, , , These are the weighting coefficients.
[0047] It should be noted that graph convolutional neural networks effectively solve the problem of spatial registration between morphological and spectral data by modeling spatial relationships.
[0048] The aforementioned multimodal data spatial registration method based on graph convolutional neural networks constructs a graph structure with pixels as nodes and spatial adjacency and feature similarity as edges. An explicit spatial correspondence between morphological and spectral data was established; the message passing mechanism of the graph convolutional layer was also demonstrated. Spatial context modeling is achieved by aggregating neighborhood node information. The specific implementation of the MLP function is as follows: First, node features are... , Sum of edge features The input vector is obtained by concatenation. Then, a nonlinear transformation is performed through a two-layer fully connected network, the first layer being... The second layer is ,in This is the weight matrix. As the bias vector, the ReLU activation function is used to introduce nonlinearity, and the Tanh activation function is used to constrain the message value to a certain value. To enhance numerical stability within the range; and They are nodes and Features By using edge features, the problem of spatial registration error accumulation caused by traditional independent processing of multimodal data is overcome.
[0049] In this embodiment of the application, in order to further improve the accuracy of variety identification, the method further includes step 5: correcting the variety identification results based on deep metric learning and generating quality grading decisions.
[0050] Step 5 includes: Step 501: Calculate the Euclidean distance between the current sample features and the features of each variety in the standard variety library using a deep metric learning model. This model is based on a multilayer perceptron architecture, including an input layer, three hidden layers, and an output layer. The input layer has a receptive dimension of [missing information]. eigenvectors The formula for calculating the hidden layer is: Output layer generation dimensional embedding vector The model employs a contrastive learning training mode and uses a triplet loss function. Training was conducted, among which For anchor point sample features, For positive sample features, The features are negative samples, and margin is the boundary parameter.
[0051] Step 502: Based on the defect type and severity, locate the preset spectral distortion compensation coefficient table and obtain the compensation coefficient. .
[0052] Step 503: Correct the distance metric using the compensation coefficient. Then, the variety similarity was recalculated.
[0053] Step 504: Based on the comprehensive defect evaluation results and the corrected variety identification results, generate the final quality grading decision according to the preset grading rules; decode the quality grading decision into specific executable results: based on the defect type weight. Severity weighting Matching degree with variety Calculate the overall quality score ,in Rice is classified into superior grades based on a quality fraction threshold. First-class products Second-class products ) and non-conforming products ( It has four levels and outputs specific defect descriptions, variety identification results, and suggested handling methods.
[0054] The aforementioned spectral distortion compensation mechanism based on defect state establishes a mapping model between defect type-severity and spectral feature changes. Predict the extent to which defects affect the original spectrum; function The specific implementation is as follows: First, the defect type is one-hot encoded to obtain a vector. Normalize the severity into a scalar Then, feature change prediction is performed through a hierarchical mapping mechanism, with the first layer being type-specific transformation. The second layer is severity modulation. Finally, the spectral change vector is obtained through nonlinear combination. ,in This is the weight matrix. For bias vectors, For the tensor product operation, a cubic polynomial term is used to capture the severity of nonlinear effects; and through compensation coefficients... Correct distance metric, where The parameter represents the intensity of the defect's influence. This mechanism effectively suppresses the interference of spectral noise caused by defects on variety identification, ensuring that rice varieties can still be accurately identified even in the presence of defects.
[0055] Technical effects of this embodiment: This implementation method obtains the reflectance distribution of different regions through spectral pre-scanning and adaptively adjusts the structured light projection parameters accordingly. This overcomes the problem of uneven phase calculation errors caused by differences in surface reflectance characteristics of different rice varieties, thus solving the technical problem of insufficient accuracy in 3D morphology reconstruction. By extracting ternary joint features of spectrum, texture, and depth through non-negative matrix factorization, and utilizing the complementarity of multi-source information, it overcomes the local information loss caused by defect shadow occlusion, solving the problem of incomplete single-modal data affecting detection accuracy. By constructing a graph structure and using a graph convolutional neural network to model spatial relationships, it overcomes the loss of spatial correspondence caused by the independent processing of morphological and spectral data in traditional methods, solving the problem of insufficient spatial registration accuracy of multimodal data. By combining deep metric learning with a spectral distortion compensation mechanism, it overcomes the spectral feature shift caused by changes in defect state, solving the problem of defects interfering with the accuracy of variety identification.
[0056] A rice processing plant's intelligent sorting production line received a batch of mixed-variety rice, requiring automated quality inspection and variety separation. This batch included multiple varieties such as Wuchang fragrant rice, Thai fragrant rice, and Northeast japonica rice, and exhibited various defects including insect infestation, cracks, and breakage. The inspection equipment was equipped with a hyperspectral imaging system in the 400-1000nm wavelength band and a three-frequency structured light projection system, and needed to perform real-time inspection at a conveyor belt speed of 0.5m / s.
[0057] Raw test data: Table 1. Raw data from hyperspectral imaging (partial bands)
[0058] Table 2. Raw data from structured light 3D measurement;
[0059] Example of core processing steps: For sample Rice_004 (which was found to have insect infestation defects), the system performs the following processing: Step 1: Using K-means clustering, the sample is divided into 4 spectral regions, where the average reflectance of the insect-infested region is... Average reflectance of normal area .
[0060] Step 2 implementation: Based on the difference in reflectivity, the projected light intensity of the insect-infested area is adjusted to... lux, normal area projected light intensity is lux. Structured light measurement reveals the depth of insect infestation. mm, area .
[0061] Step 3: Extracting geometric feature vectors (Corresponding area, perimeter, maximum depth, average depth, and volume), obtained after normalization. .
[0062] Step 4: Output probability vector of graph convolutional network: defect classification (Corresponding to intact, insect-damaged, and cracked), severity (Corresponding to mild, moderate, and severe), variety identification (Corresponding to Northeast Japonica Rice, Wuchang Daohuaxiang Rice, and Thai Jasmine Rice).
[0063] Step 5: Calculate the Euclidean distance between the deep metric learning and standard variety library. , , Insect infestation defect compensation coefficient Corrected distance .
[0064] Test result data: Table 3. Statistics of Defect Detection Results;
[0065] Table 4. Results of variety identification and quality grading;
[0066] This application example demonstrates that the method can effectively identify rice defects, accurately determine variety types, and make reasonable quality grading decisions based on comprehensive evaluation results in a real production environment, thus meeting the intelligent testing needs of the rice processing industry.
Claims
1. A method for inspecting rice based on photosensitive technology, characterized in that, Includes the following steps: Hyperspectral image data of rice samples to be tested were acquired using a visible-near-infrared hyperspectral imaging system. The reflectance distribution of each band was calculated. Based on the reflectance distribution and texture features, a clustering algorithm was used to segment the image into regions with different reflectance characteristics, and a spectral partition mask was generated. Based on the average reflectance of each region in the spectral partition mask, the optimal projected light intensity of each region is calculated. Multi-frequency sinusoidal fringe sequences are projected into the regions, and the absolute phase is obtained and the three-dimensional height distribution is calculated using a multi-frequency phase unfolding algorithm to generate three-dimensional point cloud data and identify candidate defect regions. The stripe pattern within the defect candidate region is analyzed, geometric feature vectors are extracted, and the spectral data, texture features, and depth information are jointly decomposed using the non-negative matrix factorization method to generate a multimodal feature tensor. The defect candidate region is constructed as a graph structure, and multimodal features are input as node features into the graph convolutional neural network. The neighborhood information is aggregated and the node representation is updated through the graph convolutional layer. Defect type classification, severity assessment and variety origin identification are performed respectively, and a defect-variety comprehensive evaluation matrix is output. Among them, the calculation of the optimal projected light intensity is based on the inverse relationship between the reference light intensity and the average reflectivity of the region. The multi-frequency phase unrolling algorithm obtains the absolute phase by solving for integer multiples through frequency combination. The graph convolutional layer performs weighted aggregation of node features through adjacency matrix and degree matrix.
2. The rice inspection method based on photosensitive technology according to claim 1, characterized in that, The steps for generating a spectral partition mask include: Extract the gray-level co-occurrence matrix of each band image, calculate contrast, correlation, energy and homogeneous texture features, and generate a texture feature vector set; The reflectance and texture features are preprocessed by standardization, and the feature vectors are standardized by mean and variance. Pixel grouping is achieved by minimizing the weighted sum of squared distances between pixels and cluster centers, outputting a set of cluster labels and cluster centers, and generating a spectral partition mask.
3. The rice inspection method based on photosensitive technology according to claim 1, characterized in that, The steps to acquire 3D point cloud data include: Based on the calculated optimal projected light intensity, the regional projection includes a multi-frequency sinusoidal fringe sequence with low, medium, and high frequencies. The wrap-around phase at each frequency was calculated using the four-step phase-shifting method. The four-step phase-shifting fringe images are phase-shifted images at 0 degrees, 90 degrees, 180 degrees, and 270 degrees, respectively. The absolute phase is obtained by combining low-frequency wrapped phase, mid-frequency wrapped phase, and high-frequency wrapped phase using a multi-frequency phase expansion algorithm. The three-dimensional height distribution is calculated based on the linear mapping relationship between the absolute phase and the system calibration distance, generating three-dimensional point cloud data.
4. The rice inspection method based on photosensitive technology according to claim 1, characterized in that, The steps for generating multimodal feature tensors include: The defect depth is estimated based on the location of the stripe break and the degree of deformation, where the product of the phase jump variable and the system working distance is used to determine the defect depth. Extract the geometric feature vector of the defect region, including defect area, perimeter, maximum depth, average depth, and volume; Perform maximum-minimum normalization preprocessing on the geometric eigenvectors to normalize each eigenvalue to the interval between zero and one. The multimodal data is preprocessed using a unified normalization method. The original matrix is decomposed under non-negativity constraints by minimizing the reconstruction error to obtain the basis matrix and coefficient matrix.
5. The rice inspection method based on photosensitive technology according to claim 1, characterized in that, The steps involved in graph convolutional neural network processing include: The defect candidate region is constructed as a graph structure, where nodes represent local regions and edges are constructed based on spatial adjacency and feature similarity. The geometric features of the defective region, the spectral features of the surrounding intact region, and the global texture distribution are used as node features; The node feature vectors are preprocessed by L2 normalization to ensure that all types of features participate in network computation at a uniform scale. The output layer consists of three parallel fully connected layers, which output the defect classification probability, severity probability, and variety identification probability, respectively.
6. The rice inspection method based on photosensitive technology according to claim 1, characterized in that, A multi-task loss function is used for network training. The multi-task loss function is a weighted combination of defect classification cross-entropy loss, severity classification cross-entropy loss, and variety identification cross-entropy loss.
7. The rice inspection method based on photosensitive technology according to claim 1, characterized in that, After outputting the defect-variety comprehensive evaluation matrix, the following steps are also included: Calculate the Euclidean distance between the current sample features and the features of each variety in the standard variety library using a deep metric learning model; Based on the defect type and severity, find the preset spectral distortion compensation coefficient table and obtain the compensation coefficient; The distance metric is corrected using a compensation coefficient, and the variety similarity is recalculated. Based on the combined defect evaluation results and the corrected variety identification results, the final quality grading decision is generated according to the preset grading rules.
8. The rice inspection method based on photosensitive technology according to claim 7, characterized in that, The deep metric learning model is based on a multilayer perceptron structure, including an input layer, a hidden layer, and an output layer. It adopts a contrastive learning training mode and is trained using a triplet loss function, which includes anchor sample features, positive sample features, negative sample features, and boundary parameters.
9. The rice inspection method based on photosensitive technology according to claim 7, characterized in that, Quality grading decisions include: The overall quality score is calculated based on the defect type weight, severity weight, and variety matching degree, where the sum of the defect type weight and variety matching degree weight is one. Based on the quality score threshold, rice is divided into four grades: superior, first-class, second-class, and unqualified. At the same time, it outputs specific defect descriptions, variety identification results, and suggested handling methods.
10. A rice inspection system based on photosensitive technology, used to execute the rice inspection method based on photosensitive technology as described in any one of claims 1-9, characterized in that, include: Visible-near-infrared hyperspectral imaging system, used to acquire hyperspectral image data of rice samples; A structured light projection system is used to adaptively adjust projection parameters based on a spectral partition mask and project a multi-frequency sinusoidal fringe sequence. The image processing unit is used to calculate reflectance distribution, generate spectral partition masks, and extract multimodal features; The graph convolutional neural network processing unit is used for graph structure modeling and comprehensive evaluation of multimodal features; A deep metric learning unit is used to correct the variety identification results and generate quality grading decisions.