An online detection method and system for broken rice rate of cereals based on image analysis

Abstract

The application relates to the technical field of grain detection, and discloses a kind of on-line detection method and system of grain broken rice rate based on image analysis, wherein a kind of on-line detection method of grain broken rice rate based on image analysis includes seven steps of intelligent illumination control and high-speed synchronous imaging, adaptive image enhancement based on reinforcement learning, multi-stage parallel particle detection and segmentation, multi-modal feature deep fusion, robust classification based on integrated learning, intelligent quality control and process optimization, continuous learning and knowledge updating. Clear images are obtained through multi-angle linear array cameras and stroboscopic light sources, adaptive image enhancement is carried out through reinforcement learning agents, particle detection and segmentation are carried out using a deep learning network, multi-dimensional features are extracted and accurate identification is realized through an integrated classifier, and a digital twin model is established to optimize process parameters. The application can realize accurate on-line detection of grain broken rice rate under high-speed flow state, and improve detection efficiency and accuracy.

Description

Technical Field

[0001] This invention relates to the field of grain detection technology, and more specifically, to an online detection method and system for the breakage rate of grains based on image analysis. Background Technology

[0002] The broken rice rate is a crucial indicator of rice quality, directly impacting the market value and edible quality of grains. In modern grain processing, the milling stage is the primary source of broken rice, and the settings of process parameters such as milling pressure and rotation speed directly affect the broken rice rate of the final product. Traditional methods for detecting broken rice rate mainly rely on manual sampling and offline analysis, which suffers from low efficiency, large human error, and poor real-time performance, failing to meet the demands of real-time quality control in modern industrial production.

[0003] With the rapid development of computer vision technology and image processing algorithms, automatic detection methods based on image analysis have gradually attracted attention. Existing machine vision inspection systems mostly employ static image detection methods, which face challenges such as motion blur, particle overlap, and uneven lighting when processing high-speed flowing grains, resulting in room for improvement in detection accuracy and stability. Furthermore, current technologies lack adaptability to different grain varieties, making it difficult to cope with complex and ever-changing industrial environments.

[0004] Although recent research on related technologies has made progress in some aspects, there are still technical problems such as poor adaptability to detection environment, insufficient real-time performance, lack of adaptability to multiple varieties, and lack of feedback control mechanism. There is an urgent need to develop an online detection technology for grain broken rice rate that is adaptable to industrial environment, has real-time detection capability, supports multi-variety adaptation, and integrates feedback control. Summary of the Invention

[0005] This invention provides an online detection method and system for grain broken rice rate based on image analysis, which solves the technical problems of weak detection capability and limited detection varieties in related technologies.

[0006] This invention provides an online detection method for the broken rice rate of grains based on image analysis, comprising the following steps: Grain particle flow images are acquired by a multi-angle camera array and a stroboscopic light source system. Multi-angle light source configuration and polarization filtering technology are used to eliminate reflective interference and output a particle flow image sequence. Based on granular image sequences, a reinforcement learning agent is used to select the optimal combination of image enhancement parameters and output an enhanced image. Perform particle detection and segmentation on the enhanced image, and output particle region and contour information; Based on granular region and contour information, geometric, texture, color, and shape features are extracted; multimodal features are fused to output a deep fusion feature vector; The deep fusion feature vector is input into the ensemble classifier to obtain the particle classification result; The real-time broken rice rate is calculated based on the particle classification results, and a predictive model is used to predict the rice milling trend. Based on the predicted rice milling trend, the operating parameters of the rice milling machine are optimized and adjusted in real time. An incremental learning database is established to collect historical data for testing, and knowledge distillation technology is used to update the model to achieve continuous learning and knowledge updating.

[0007] In a preferred embodiment, the step of acquiring the grain particle flow image further includes using laser line scanning imaging technology: An 808 nm near-infrared laser is used as the light source, combined with a CMOS linear array sensor for imaging. The laser has a power of 50 watts, a beam width of 2 millimeters, and a scanning frequency of 2 kilohertz. The laser imaging system uses the triangulation principle to simultaneously acquire the geometric shape and surface texture information of grain particles.

[0008] In a preferred embodiment, the multi-angle camera array and stroboscopic light source system includes: Three sets of linear array cameras are installed in the middle section of the conveying pipeline, including a main camera that points vertically downward from the top and auxiliary cameras that are tilted at 45 degrees on both sides, forming a triangular imaging geometry. The main camera has a resolution of 4096×2048 pixels, the auxiliary camera has a resolution of 2048×1024 pixels, and the cameras are synchronized by hardware, with the synchronization accuracy controlled within 1 microsecond. It is equipped with a ring-shaped strobe LED light source array, which contains 12 independently controlled light source units, each with a power of 200W and a color temperature of 6500K. The light source frequency is precisely synchronized with the camera frame rate, and the strobe frequency is 1000Hz.

[0009] In a preferred embodiment, the reinforcement learning agent includes: Deploy a multi-source environmental sensor network, including illuminance sensors, dust concentration detectors, vibration accelerometers, and temperature and humidity sensors; Construct a reinforcement learning agent based on a deep Q-network. The state space includes environmental parameters and image quality indicators, and the action space includes continuous parameters such as contrast adjustment, denoising intensity, and color correction. The agent selects the optimal combination of image enhancement parameters based on the current state, including gamma correction coefficients, Gaussian filter kernel size, bilateral filter parameters, and histogram equalization intensity.

[0010] In a preferred embodiment, the particle detection and segmentation of the enhanced image includes: A lightweight YOLOv8 network is used for fast particle detection and localization. The network input size is 640×640 pixels. The backbone network uses CSPDarknet53 and the neck network uses PANet structure. For the precise segmentation of the particle regions located in the coarse detection stage, an improved Mask R-CNN network is used, with ResNet-101 as the backbone network and FPN structure as the feature pyramid network. Temporal information constraints are used to eliminate false detections caused by changes in illumination and particle motion. A particle tracking mechanism is established, Kalman filtering is used to predict particle trajectories, and the Hungarian algorithm is combined for data association.

[0011] In a preferred embodiment, the fused multimodal features include: For each segmented particle, calculate the geometric feature parameters, extract the particle outline, and simplify the outline using the Douglas-Peucker algorithm; calculate the basic geometric features such as area, perimeter, length and width of the minimum bounding rectangle, centroid coordinates, and principal axis direction angle. Texture features are calculated based on the gray-level co-occurrence matrix, including contrast, correlation, energy, homogeneity, and entropy statistics, with parameter combinations of four directions and three distances set. Convert the RGB image to HSV and Lab color spaces, calculate the mean, standard deviation, skewness, and kurtosis statistical characteristics of each channel, and calculate the color histogram characteristics. Design a multi-branch feature fusion network with four branches that process geometric, texture, color, and shape features respectively. The correlation between different modal features is calculated through a cross-attention mechanism.

[0012] In a preferred embodiment, the ensemble classifier includes: Three different types of classifiers were constructed: a lightweight convolutional neural network, a Transformer classifier, and an XGBoost gradient boosting tree. The CNN classifier adopted the MobileNetV3 architecture and contained 13 inverse residual blocks. Based on the feature distribution and classification difficulty of the input samples, the weights of each classifier are dynamically adjusted. A confidence-based weight allocation method is adopted, with classifiers with higher confidence scores being assigned greater weights. The final classification decision is made by weighted voting. The voting weight of each classifier is determined by its dynamic weight and historical accuracy. Broken rice is further subdivided into three grades according to the length of the grain.

[0013] In a preferred embodiment, the real-time optimization and adjustment of the rice milling machine's operating parameters includes: A digital twin model of the rice milling process was established to simulate the complex relationship between the process parameters of rice milling pressure, rotation speed, and feed rate and the broken rice rate. The model adopts a deep neural network structure, which includes an input layer, three hidden layers and an output layer. Within a sliding time window, the number of various types of particles is counted, the real-time broken rice rate is calculated, and a weighted average method is used to process historical data, with recent data having a higher weight. The model predictive control algorithm is used to predict the future trend of broken rice rate and optimize the rice milling process parameters. The prediction domain is set to 10 sampling periods and the control domain is set to 5 sampling periods. Based on the output of the predictive control algorithm, the pressure and speed parameters of the rice milling machine are automatically adjusted.

[0014] In a preferred embodiment, the continuous learning and knowledge updating includes: An incremental learning database is constructed, using a hierarchical storage structure and indexed by variety, time, and process parameters. Each batch of new samples is processed at a time, with the batch size set to 32. The knowledge distillation technique is used to transfer the knowledge of the complex teacher model to the lightweight student model. The knowledge distillation process includes two parts: hard label loss and soft label loss. The teacher model is a complete ensemble classifier, and the student model is a single lightweight network. An elastic weight consolidation algorithm is used to prevent catastrophic forgetting. The Fisher information matrix of parameter importance is calculated, and stronger regularization constraints are imposed on important parameters. The pruning ratio is 30%, and the quantization precision is 8-bit integer. Establish an experience playback mechanism to intersperse historical data during the training of new data, maintain the memory of old knowledge, and adaptively adjust the model update frequency according to changes in detection accuracy. When the accuracy drops by more than 5%, the model is triggered to update.

[0015] In a preferred embodiment, an image analysis-based online grain breakage rate detection system is used to perform the above-described image analysis-based online grain breakage rate detection method, comprising: The image acquisition module is used to acquire images of grain particle flow through a multi-angle camera array and a strobe light source system. It uses a multi-angle light source configuration and polarization filtering technology to eliminate reflective interference and outputs a sequence of particle flow images. The image enhancement module is used to select the optimal combination of image enhancement parameters based on the granular flow image sequence and output the enhanced image. The particle detection module is used to detect and segment particles in the enhanced image, and output particle region and contour information. The feature extraction module is used to extract geometric, texture, color, and shape features based on granular region and contour information; it also fuses multimodal features to output a deep fusion feature vector. The classification and recognition module is used to input the deep fusion feature vector into the ensemble classifier to obtain the particle classification result; The control optimization module is used to calculate the real-time broken rice rate based on the particle classification results, and to predict the rice milling trend using a predictive model; based on the predicted rice milling trend, the operating parameters of the rice milling machine are optimized and adjusted in real time. The learning and updating module is used to build an incremental learning database to collect historical data and update the model using knowledge distillation technology, thereby achieving continuous learning and knowledge updating.

[0016] The beneficial effects of this invention are as follows: Employing multi-angle high-speed imaging and reinforcement learning-based adaptive image enhancement techniques, the system effectively addresses motion blur and image quality issues in high-speed flowing conditions. Accurate particle detection and segmentation are achieved through deep learning networks, combined with multimodal feature fusion and ensemble learning classification methods, enabling precise identification of whole rice grains, large broken rice grains, medium broken rice grains, and small broken rice grains. Furthermore, the system maintains stable detection performance under various environmental conditions, demonstrating strong robustness. Breaking through the limitations of traditional offline testing, this system enables continuous online testing at a high-speed conveying rate of 2 to 3 tons per minute. The test results are fed back to the rice milling machine control system in real time, allowing for automatic adjustment and optimization of process parameters based on the test results. By establishing a digital twin model and employing model predictive control algorithms, the system can predict future broken rice rate trends and adjust process parameters in advance, forming a complete closed-loop control system that improves production efficiency and product quality consistency. Attached Figure Description

[0017] Figure 1 This is a flowchart of an online detection method for grain broken rice rate based on image analysis according to the present invention; Figure 2 This is a block diagram of an online grain breakage rate detection system based on image analysis according to the present invention; Figure 3 This is a diagram showing the effect of optimizing the broken rice rate of the present invention. Detailed Implementation

[0018] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, some features described in the examples may be combined in other examples.

[0019] At least one embodiment of the present invention discloses an online detection method for the broken rice rate of grains based on image analysis, such as... Figure 1As shown, it includes the following steps: Step 100: Acquire grain particle flow images through a multi-angle camera array and stroboscopic light source system, eliminate reflective interference by using multi-angle light source configuration and polarization filtering technology, and output particle flow image sequence; Step 101: Deployment of a multi-angle camera array; three sets of line-scan cameras are installed in the middle section of the conveying pipe, including a main camera pointing vertically downwards from the top and auxiliary cameras tilted at 45 degrees on both sides, forming a triangular imaging geometry. The main camera uses a resolution of 4096×2048 pixels and a frame rate of 1000fps to ensure the capture of details of rice grains moving at high speed. The auxiliary cameras have a resolution of 2048×1024 pixels and a frame rate of 1000fps, used to acquire side morphological information of the grains. Hardware synchronization is used between the cameras, with synchronization accuracy controlled within 1 microsecond.

[0020] Step 102, Strobe Light Source System Design: A ring-shaped strobe LED light source array is configured, containing 12 independently controlled light source units, each with a power of 200W and a color temperature of 6500K. The light source frequency is precisely synchronized with the camera frame rate, with a strobe frequency of 1000Hz and a single flash duration of 50 microseconds. Pulse width modulation (PWM) technology is used to control the light intensity, which is adjustable from 20% to 100%, with a response time of less than 10 microseconds.

[0021] Step 103, Adaptive Illumination Control Algorithm: Based on the ambient light sensor and rice color detection results, dynamically adjust the brightness and color temperature of the strobe light source. The algorithm employs closed-loop feedback control, with the objective function being a weighted combination of image contrast and saturation. The illumination control strategy automatically switches according to the rice variety: warm-toned lighting for indica rice, neutral-toned lighting for japonica rice, and cool-toned lighting for glutinous rice. The illumination control output is calculated by subtracting the product of the contrast weighting coefficient and the target contrast, and the product of the saturation weighting coefficient and the target saturation, from the sum of these products.

[0022] It outputs stable and clear multi-angle grain flow image sequences with motion blur controlled within 0.5 pixels and an image signal-to-noise ratio of over 40dB.

[0023] Furthermore, laser line scanning imaging technology is employed, using an 808nm near-infrared laser as the light source, in conjunction with a CMOS linear array sensor for imaging. The laser power is 50W, the beam width is 2mm, and the scanning frequency is 2000Hz. This technology is more sensitive to the surface texture of rice, capable of detecting minute cracks and surface defects. The laser imaging system uses the triangulation principle to simultaneously acquire two-dimensional images and three-dimensional morphological information of the particles. The laser safety level is Class 3R, equipped with a safety shield and an emergency stop device. The purpose of using laser line scanning imaging technology is to improve the detection accuracy of microscopic defects on the surface of rice. Compared with traditional visible light imaging, laser imaging can penetrate the fine dust layer on the surface of rice to obtain clearer surface texture information, thereby improving the detection accuracy of surface cracks. It is particularly suitable for detecting micro-cracks caused by mechanical stress, providing a more accurate basis for predicting broken rice.

[0024] Step 200: Based on the granular flow image sequence, a reinforcement learning agent is used to select the optimal combination of image enhancement parameters and output the enhanced image; Step 201: Environmental parameter acquisition and preprocessing; deploy a multi-source environmental sensor network, including a light intensity sensor, a dust concentration detector, a vibration accelerometer, and a temperature and humidity sensor. The light intensity sensor has a measurement range of 0 to 100,000 Lux and an accuracy of ±2%; the dust sensor uses the laser scattering principle, detects particles from 0.3 to 10 μm, and has a concentration accuracy of ±3%; the vibration sensor has a frequency response range of 1 to 1000 Hz and an acceleration range of ±50 g. Sensor data is acquired at a frequency of 100 Hz and, after Kalman filtering, is used as the state input for the reinforcement learning agent.

[0025] Step 202, Reinforcement Learning Agent Design: Construct a reinforcement learning agent based on a deep Q-network. The state space includes environmental parameters and image quality metrics, while the action space includes 12 continuous parameters such as contrast adjustment, denoising intensity, and color correction. The network structure adopts a two-stream design, with the environmental parameter branch being a fully connected network and the image feature branch being a convolutional neural network. Finally, feature fusion is performed through an attention mechanism.

[0026] Step 203: Adaptive enhancement strategy execution; the agent selects the optimal combination of image enhancement parameters based on the current state, including gamma correction coefficients, Gaussian filter kernel size, bilateral filter parameters, and histogram equalization intensity. The enhancement process uses GPU parallel computing, supporting real-time processing of 1000fps image streams. Specifically, the gamma correction coefficients range from 0.5 to 2.0, the Gaussian filter kernel size ranges from 3×3 to 9×9 pixels, the spatial domain standard deviation of the bilateral filter ranges from 5 to 25, the color domain standard deviation ranges from 20 to 80, and the histogram equalization intensity uses an adaptive contrast-limited histogram equalization method, limiting the contrast range to 2.0 to 8.0.

[0027] in, Indicates time The reward value ranges from 1 to 1; This represents the structural similarity index between the enhanced image and the target image, with a value ranging from 0 to 1. A larger value indicates better image quality. This indicates the signal-to-noise ratio of the enhanced image, measured in dB, with a typical value of 20 to 50 dB. This indicates the processing time in milliseconds, with a target value of less than 1 millisecond. This represents the enhanced image; Represents the target reference image; This represents the image quality weight, with a value of 0.6. This represents the noise control weight, with a value of 0.3. This represents the time constraint weight, with a value of 0.1.

[0028] Output an optimally enhanced image that adapts to the environment.

[0029] Furthermore, a conditional generative adversarial network (GAN) is constructed using an image enhancement method based on generative adversarial networks (GANs). The generator network is responsible for image enhancement, and the discriminator network evaluates the enhancement effect. The generator adopts a U-Net architecture, including an encoder, a decoder, and skip connections. The encoder uses a residual block structure, with each residual block containing two convolutional layers, a batch normalization layer, and a ReLU activation function. The decoder uses transposed convolutions for upsampling, and skip connections are used to preserve detail information. The discriminator adopts a PatchGAN structure, outputting a 70×70 discrimination result matrix. The total generator loss is calculated by a weighted combination of adversarial loss, content loss, and perceptual loss, where the content loss has a weight of 100 and the perceptual loss has a weight of 10. The purpose of adopting an image enhancement method based on generative adversarial networks is to achieve more intelligent image quality improvement under complex environmental conditions. Through the adversarial training mechanism, the generator can learn enhancement strategies that are more in line with human visual perception. Compared with traditional filtering methods, it can improve the clarity of details while maintaining the naturalness of the image. It performs particularly well in low light and high noise environments, and improves the accuracy of subsequent particle detection.

[0030] Step 300: Perform particle detection and segmentation on the enhanced image, and output particle region and contour information; Step 301, Coarse Detection Stage—Rapid Target Localization: A lightweight YOLOv8 network is used for rapid detection and localization of rice grains. The network input size is 640×640 pixels, the backbone network uses CSPDarknet53, the neck network uses a PANet structure, and the detection head adopts a decoupled design. The network is trained on a self-built rice dataset containing 500,000 labeled images covering 10 major rice varieties.

[0031] Step 302, Fine Detection Stage—Accurate Instance Segmentation: The particle regions located in the coarse detection stage are precisely segmented using an improved Mask R-CNN network. The backbone network uses ResNet-101, the feature pyramid network adopts an FPN structure, the RPN network generates candidate regions, the classification head and regression head are responsible for category determination and bounding box regression, respectively, and the segmentation head outputs pixel-level masks. The network has been optimized for the characteristics of rice particles, including adding a small target detection branch, introducing deformable convolutions, and adding an edge enhancement module.

[0032] Step 303: Temporal consistency constraints and false detection elimination; temporal information constraints are used to eliminate false detections caused by factors such as illumination changes and particle motion. A particle tracking mechanism is established, using Kalman filtering to predict particle trajectories and combining it with the Hungarian algorithm for data association. For inconsistent detection results appearing in consecutive frames, a temporal voting mechanism is used for correction. Temporal consistency loss is measured by calculating the sum of the L2 norm distances between the segmentation masks of adjacent frames, with the temporal window length set to 8 frames.

[0033] Outputs precisely segmented rice grain regions and their outlines.

[0034] Furthermore, a Transformer-based segmentation network is employed, using the DETR (DEtectionTRansformer) architecture for end-to-end object detection and segmentation. The encoder adopts a standard Transformer structure, containing six encoder layers, each including a multi-head self-attention mechanism and a feedforward network. The decoder also contains six layers, employing a cross-attention mechanism to focus on image features. Position encoding uses sine and cosine encoding, supporting inputs of arbitrary sizes. The query vector is initialized with learnable parameters, with a quantity set to 300. Training uses the Hungarian matching algorithm to calculate the loss, including classification loss, bounding box regression loss, and segmentation mask loss. The purpose of using a Transformer-based segmentation network is to capture the global spatial relationship between rice grains using a self-attention mechanism, overcoming the problem of limited receptive field in traditional convolutional networks. It is particularly suitable for handling densely arranged and mutually occluded grain scenes, improving the segmentation accuracy of overlapping grains, and reducing missegmentation caused by grain adhesion, thus providing more accurate grain boundary information for subsequent feature extraction.

[0035] Step 400: Based on granular region and contour information, extract geometric, texture, color, and shape features; fuse multimodal features and output a deep fusion feature vector; Step 401, Geometric Feature Extraction: For each segmented rice grain, a series of geometric feature parameters are calculated. The grain outline is extracted and simplified using the Douglas-Peucker algorithm. Basic geometric features such as area, perimeter, length and width of the minimum bounding rectangle, centroid coordinates, and principal axis direction angle are calculated. Due to the significant differences in the dimensions and numerical ranges of different geometric features, standardization preprocessing is required: area features are normalized by dividing by the total number of pixels in the image; perimeter features are normalized by dividing by the length of the image diagonal; angle features (such as the principal axis direction angle) are normalized to the range of 0 to 1 by dividing by 360 degrees; coordinate features are normalized by subtracting the image center coordinates and then dividing by the image size. Further calculations of shape descriptors such as roundness, rectangularity, elongation, and convexity are performed. These features are already dimensionless parameters in the range of 0 to 1 and require no additional preprocessing.

[0036]

[0037] in, The value represents the roundness, ranging from 0 to 1. The closer the value is to 1, the closer the shape is to a circle. The roundness of a whole rice grain is usually 0.6 to 0.8. This indicates the particle area, in pixels, with a typical value of 2000 to 8000 pixels; This indicates the perimeter of the grain, in pixels, with a typical value of 200 to 400 pixels.

[0038]

[0039] in, This represents the elongation, with a value ranging from 0 to 1. A larger value indicates finer and longer particles. This indicates the length of the main axis, which is the maximum length of the particle; This indicates the length of the secondary axis, which is the maximum width perpendicular to the direction of the primary axis.

[0040] Step 402, Texture Feature Extraction: Texture features are calculated based on the gray-level co-occurrence matrix, including statistics such as contrast, correlation, energy, homogeneity, and entropy. Parameter combinations are set for four directions (0°, 45°, 90°, 135°) and three distances (1, 2, 3 pixels). Simultaneously, a local binary mode algorithm is used to extract rotation-invariant texture features, setting three scales with radii of 1, 2, and 3 pixels, and sampling numbers of 8, 16, and 24 points respectively. Texture contrast is obtained by summing the products of the squares of gray-level differences and their corresponding probabilities, and texture entropy is measured by the negative logarithm-weighted sum of the probabilities of each gray level to measure texture complexity.

[0041] Step 403, Color Feature Extraction: The RGB image is converted to HSV and Lab color spaces, and statistical features such as mean, standard deviation, skewness, and kurtosis of each channel are calculated. Focusing on the color characteristics of rice grains, the hue information of the H channel and the brightness information of the L channel are prioritized. Before feature extraction, the color data needs to be preprocessed: the H channel values ​​in the HSV space are normalized, mapping angle values ​​from 0 to 360 degrees to the 0 to 1 range; the L channel values ​​in the Lab space are standardized, normalizing brightness values ​​from 0 to 100 to the 0 to 1 range; and the a and b channel values ​​are zero-mean processed to eliminate the influence of color deviation. Simultaneously, the color histogram features are calculated, quantizing each channel into 32 bins to form a 96-dimensional color histogram feature vector.

[0042] Step 404, Shape Feature Extraction: Fourier descriptors are used to characterize the shape features of the particles. Fourier transform is performed on the contours, and the first 32 low-frequency coefficients are taken as shape descriptors. For rice particles with complex shapes, Zernike moment features are introduced, calculated up to order 12, with a total of 49 feature parameters.

[0043] Step 405, Cross-Attention Feature Fusion: Design a multi-branch feature fusion network with four branches processing geometric, texture, color, and shape features respectively. Each branch uses a fully connected network for feature transformation, and then calculates the correlation between different modal features through a cross-attention mechanism to generate an attention weight matrix.

[0044]

[0045] in, This represents the fused feature vector, with a dimension of 64. Indicates the first Feature vectors of each modality; This represents the total number of feature modes, with a value of 4. Indicates the first Attention weights for each modality.

[0046]

[0047] in, Indicates the first Attention weights for each modality, ranging from 0 to 1, with the sum of all weights being 1; Indicates the first The weight matrix of each modality has dimensions of . ; Indicates the first The original dimensions of each modal feature; Represents an exponential function; Indicates the first The weight matrix of each modality; Indicates the first Feature vectors of each modality This represents the total number of modes.

[0048] Output a 64-dimensional deep fusion feature vector for each rice grain.

[0049] Furthermore, a feature learning method based on graph neural networks is adopted, treating each rice grain as a node in a graph, and constructing edges for the spatial relationships between grains, forming a rice grain graph structure. Node features include the geometric, texture, and color features of the grains, while edge features include relational features such as distance, relative position, and size ratio between grains. A graph convolutional network is used for feature learning, consisting of three graph convolutional layers, each followed by batch normalization and a ReLU activation function. Graph pooling employs a TopK pooling strategy to retain the most important nodes. Finally, a global feature representation is generated through a graph-level readout function. The purpose of adopting a feature learning method based on graph neural networks is to make full use of the spatial adjacency relationships and mutual influences between rice grains. By modeling the graph structure, it can capture the contextual information that traditional independent feature extraction methods ignore. It is particularly suitable for analyzing the overall distribution characteristics of grain groups, which can improve the classification accuracy and enhance the model's adaptability to changes in grain density and arrangement patterns.

[0050] Step 500: Input the deep fusion feature vector into the ensemble classifier to obtain the particle classification result; Step 501, Diverse Classifier Construction; Construct three different types of classifiers: a lightweight convolutional neural network, a Transformer classifier, and an XGBoost gradient boosting tree. Before inputting the classifiers, the 64-dimensional fused features need to be preprocessed: for the CNN and Transformer classifiers, the Z-score normalization method is used, i.e. ,in and These are the mean and standard deviation on the training set, respectively; for the XGBoost classifier, the Min-Max normalization method is used, i.e. The features are scaled to the range of 0 to 1. The CNN classifier uses the MobileNetV3 architecture, contains 13 inverse residual blocks, and has only 2.7M parameters; the Transformer classifier uses the Vision Transformer structure, divides the feature vector into 8 patches, has an embedding dimension of 512, and contains 6 Transformer blocks; XGBoost uses 500 decision trees, with a maximum depth of 6 and a learning rate of 0.1.

[0051] Step 502, Dynamic Weight Allocation Strategy: The weights of each classifier are dynamically adjusted based on the feature distribution and classification difficulty of the input samples. A confidence-based weight allocation method is used, assigning greater weights to classifiers with higher confidence scores. A sample difficulty assessment mechanism is also introduced, favoring lightweight classifiers for simple samples and employing more accurate deep networks for complex samples.

[0052]

[0053] in, Represents classifier The dynamic weights for sample X range from 0 to 1, and the sum of all weights is 1. Represents classifier The confidence level ranges from 0 to 1, with a higher value indicating a higher confidence level. This represents a temperature parameter used to adjust the sharpness of the weight distribution; in this embodiment, it is set to 2.0. This represents the total number of classifiers, with a value of 3. Represents an exponential function; This represents the confidence level of classifier j; This represents the input sample.

[0054]

[0055] Represents classifier For the sample The confidence level ranges from 0 to 1, with a higher value indicating a higher confidence level. Represents classifier The maximum predicted probability for all categories; Represents classifier Category The predicted probability, with a value ranging from 0 to 1; This indicates the total number of categories, with a value of 4 (whole rice, large broken rice, medium broken rice, small broken rice). This indicates a category index.

[0056] Step 503, Weighted ensemble decision mechanism: A weighted voting method is used for the final classification decision, and the voting weight of each classifier is determined by its dynamic weight and historical accuracy. For boundary samples, a rejection mechanism is introduced. When the maximum confidence of the ensemble classifier is lower than the threshold, the sample is marked as an uncertain category and submitted for manual review.

[0057]

[0058] in, Indicates the final category The predicted probability, with a value ranging from 0 to 1; Represents classifier For the sample The dynamic weights range from 0 to 1, and the sum of all weights is 1. Represents classifier Category The predicted probability, with a value ranging from 0 to 1; This represents the total number of classifiers, with a value of 3. Indicates a category index; This represents the input sample.

[0059] Step 504: Further subdividing broken rice into three grades based on rice length: large broken rice (length ≥ 75% of whole rice length), medium broken rice (length between 50% and 75% of whole rice length), and small broken rice (length < 50% of whole rice length). Length measurement is based on the smallest bounding rectangle of the grain, using sub-pixel precision calculation, achieving a measurement accuracy of 0.1mm.

[0060] Output the classification result and confidence score for each particle.

[0061] Furthermore, a classification strategy selection method based on deep reinforcement learning is adopted to model the classification task as a Markov decision process, where the agent selects the optimal classification strategy based on the current feature state. The state space contains feature vectors and historical classification results, while the action space represents combinations of different classifiers. The reward function is designed as a trade-off between classification accuracy and computational complexity, encouraging the selection of more computationally efficient classification strategies while ensuring accuracy. The PPO algorithm is used for strategy optimization, with an Actor-Critic network structure. The Actor network outputs the policy probability distribution, and the Critic network estimates the state value function. The purpose of adopting a classification strategy selection method based on deep reinforcement learning is to achieve adaptive optimization of the classification strategy. It dynamically selects the most suitable classifier combination according to different sample features and environmental conditions, avoids the performance bottleneck of fixed strategies in complex scenarios, improves the overall classification efficiency, and reduces the consumption of computing resources while ensuring accuracy. It is particularly suitable for handling mixed detection scenarios of multiple varieties of rice.

[0062] Step 600: Calculate the real-time broken rice rate based on the particle classification results, and use a prediction model to predict the rice milling trend; based on the predicted rice milling trend, optimize and adjust the operating parameters of the rice milling machine in real time. Step 601, Digital Twin Model Construction: A digital twin model of the rice milling process is established to simulate the complex relationship between milling parameters such as milling pressure, rotation speed, and feed rate and the broken rice rate. The model adopts a deep neural network structure, including an input layer, three hidden layers, and an output layer. The number of neurons in the hidden layers are 128, 64, and 32, respectively. The ReLU activation function is used, and the Sigmoid function is used in the output layer to map the broken rice rate to the 0-1 range.

[0063] Step 602: Real-time broken rice rate statistics; within a sliding time window (30 seconds), the number of various types of grains is counted to calculate the real-time broken rice rate. A weighted average method is used to process historical data, with more recent data having higher weight. Simultaneously, the trend and standard deviation of the broken rice rate are calculated to assess production stability.

[0064] in, This represents the weighted percentage of broken rice over time t, with a value ranging from 0 to 100%. This represents the number of broken rice grains at time i, including the total number of large, medium, and small broken rice grains. This represents the total number of rice grains at time i; The weights for time t are expressed in an exponentially decaying form. The value ranges from 0 to 1; This indicates the size of the time window; in this embodiment, it is set to 30 sampling points. This represents the attenuation coefficient, with a value of 0.1.

[0065] Step 603, Model Predictive Control Algorithm: A model predictive control algorithm is used to predict the future broken rice rate trend and optimize the rice milling process parameters. The prediction domain is set to 10 sampling periods, and the control domain is set to 5 sampling periods. The objective function includes two parts: tracking performance and control effort, which are balanced by a weight matrix. Constraints include physical limitations and rate-of-change limitations on the process parameters.

[0066] in, This represents the predictive control objective function, used to measure control performance. Indicates time The reference trajectory, i.e. the target broken rice rate, is usually set to 4.0%, and needs to be normalized to the range of 0 to 1 to ensure consistency with the dimensions of the predicted output. Indicates time The predicted output, i.e. the predicted broken rice rate, also needs to be normalized to the range of 0-1. Indicates time The control increments include the adjustment amounts of rice milling pressure and speed. Since the pressure unit is MPa and the speed unit is rpm, the two have different dimensions and need to be standardized separately. The pressure adjustment amount is divided by the maximum pressure value of 1.0 MPa, and the speed adjustment amount is divided by the maximum speed value of 400 rpm. This indicates the length of the prediction domain, with a value of 10. This indicates the length of the control domain, with a value of 5. This indicates the output weight matrix, with diagonal elements having a value of 1.0; This represents the control weight matrix, with diagonal elements having a value of 0.1. This represents the weighted quadratic norm, specifically calculated as a vector and a weight matrix. Quadratic form operations, i.e. , used to measure the weighted sum of squares of tracking error; This represents the weighted quadratic norm, specifically calculated as a vector and a weight matrix. Quadratic form operations, i.e. This is used to measure the weighted sum of squares of control effort and prevent the control signal from being too large.

[0067] Step 604: Automatic adjustment of process parameters; based on the output of the predictive control algorithm, the pressure and speed parameters of the rice milling machine are automatically adjusted. The adjustment range for rice milling pressure is ±20%, and the adjustment range for speed is ±15%, with adjustment steps of 2% and 1.5% respectively. A gradual strategy is adopted for parameter adjustment to avoid the impact of large changes on production stability.

[0068] Output optimized rice milling process parameter adjustment instructions and predicted trends in broken rice rate.

[0069] Furthermore, a deep reinforcement learning-based process parameter optimization method is adopted to model the process parameter optimization problem as a continuous control task, and the DDPG algorithm is used for parameter tuning. The state space includes information such as the current broken rice rate, historical trends, and equipment status. The broken rice rate needs to be normalized to the range of 0 to 1, the historical trend is processed using a moving average and Z-score normalization, and the equipment status includes sensor data such as temperature, humidity, and vibration, which need to be normalized separately due to their different units. The action space consists of the adjustment amounts of milling pressure and rotation speed. The pressure adjustment range is [-0.2, 0.2] MPa, and the rotation speed adjustment range is [-60, 60] rpm. Both need to be normalized to the range of [-1, 1] to ensure the stability of network training. The reward function is designed as the negative value of the broken rice rate deviation, and a smoothness constraint is added to avoid frequent adjustments. The Actor network outputs a deterministic policy, and the Critic network evaluates the value of the state-action pairs. An experience replay mechanism is used to break data correlation, and the target network is used to stabilize the training process. The purpose of adopting a process parameter optimization method based on deep reinforcement learning is to achieve a more intelligent adaptive control strategy. By continuously interacting with the environment, it learns the optimal parameter adjustment strategy. Compared with traditional PID control and model predictive control, it can better handle nonlinear and time-varying characteristics, improve the control accuracy of broken rice rate, and has stronger robustness and adaptability. It is particularly suitable for mixed processing scenarios of multiple rice varieties.

[0070] Step 700: Establish an incremental learning database to collect historical detection data, and use knowledge distillation technology to update the model to achieve continuous learning and knowledge updating; Step 701: Incremental Learning Database Construction; Establish an incremental learning database containing samples of multiple rice varieties, and periodically collect detection data under new varieties and new processing conditions. The database adopts a hierarchical storage structure and is indexed by variety, time, and process parameters. Data preprocessing includes outlier detection, data cleaning, and feature normalization.

[0071] Step 702, Knowledge Distillation Model Compression: The knowledge distillation technique is used to transfer the complex knowledge from the teacher model to a lightweight student model, reducing computational complexity while maintaining accuracy. The teacher model is a complete ensemble classifier, and the student model is a single lightweight network. The distillation process includes two parts: hard-label loss and soft-label loss.

[0072]

[0073] in, This represents the total knowledge distillation loss and is used to guide students' online learning. Represents the cross-entropy loss function; Representing the actual label, using one-hot encoding; This represents the softmax activation function, which transforms logits into a probability distribution. This represents the logits output of the teacher network; This represents the logits output of the student network; This represents the distillation temperature parameter, which controls the degree of softening. In this embodiment, the value is 4.0. The value represents the trade-off between hard label and soft label loss; in this embodiment, it is set to 0.7. This represents the softmax output of the teacher network after temperature scaling. This represents the softmax output of the student network after temperature scaling.

[0074] Step 703: Implementation of the continuous learning framework; employing an elastic weight consolidation algorithm to prevent catastrophic forgetting and protect important parameters from significant modification when learning new tasks. A Fisher information matrix of parameter importance is calculated, imposing stronger regularization constraints on important parameters. Simultaneously, an experience replay mechanism is employed, interspersing historical data during new data training to maintain the retention of old knowledge.

[0075] Step 704, Model Performance Monitoring and Update: Establish a model performance monitoring mechanism to track key indicators such as detection accuracy, processing speed, and resource consumption in real time. When performance degrades beyond a threshold, the model update process is automatically triggered. Update strategies include online fine-tuning, model replacement, and parameter reset, with the optimal strategy automatically selected based on performance evaluation results.

[0076] It outputs continuously optimized detection models and an ever-improving knowledge base, with model performance gradually improving over time.

[0077] Furthermore, a meta-learning approach is employed to achieve rapid adaptation to new rice varieties. A basic model capable of quickly adapting to new rice varieties is trained using a model-independent meta-learning algorithm. The meta-learning process comprises two phases: meta-training and meta-testing. During the meta-training phase, the basic model is trained on tasks involving multiple different rice varieties, learning general feature representations and adaptation strategies. The feature data from different varieties requires standardization to eliminate scale differences between varieties; specifically, the Z-score standardization method is used. , Where is the mean, The standard deviation is used. In the meta-testing phase, rapid adaptation is performed on a small number of samples of the new variety. The features of the new variety need to be normalized using the global mean and standard deviation calculated in the meta-training phase to ensure the consistency of data distribution. A first-order MAML algorithm is adopted, achieving rapid adaptation through two layers of gradient updates. The first gradient layer is used for task-specific parameter updates, and the second gradient layer is used for meta-parameter optimization. The gradient update uses the Adam optimizer, and the learning rate needs to be adaptively adjusted according to the task complexity. The initial learning rate is set to 0.001 and dynamically adjusted using a cosine annealing strategy. The purpose of using meta-learning to achieve rapid adaptation to new varieties is to solve the problem that traditional deep learning models require a large amount of labeled data and long training time when facing new rice varieties. Meta-learning can quickly adapt with only a small number of new variety samples, shortening the adaptation time from several days to several hours, while maintaining detection accuracy and improving the practicality and promotion value of the system.

[0078] In one embodiment of the present invention, an online detection system for the breakage rate of grains based on image analysis is provided, such as... Figure 2 As shown, it includes: The image acquisition module is used to acquire images of grain particle flow through a multi-angle camera array and a strobe light source system. It uses a multi-angle light source configuration and polarization filtering technology to eliminate reflective interference and outputs a sequence of particle flow images. The image enhancement module is used to select the optimal combination of image enhancement parameters based on the granular flow image sequence and output the enhanced image. The particle detection module is used to detect and segment particles in the enhanced image, and output particle region and contour information. The feature extraction module is used to extract geometric, texture, color, and shape features based on granular region and contour information; it also fuses multimodal features to output a deep fusion feature vector. The classification and recognition module is used to input the deep fusion feature vector into the ensemble classifier to obtain the particle classification result; The control optimization module is used to calculate the real-time broken rice rate based on the particle classification results, and to predict the rice milling trend using a predictive model; based on the predicted rice milling trend, the operating parameters of the rice milling machine are optimized and adjusted in real time. The learning and updating module is used to build an incremental learning database to collect historical data and update the model using knowledge distillation technology, thereby achieving continuous learning and knowledge updating.

[0079] In one embodiment of the present invention, an application example is provided: The verification was conducted on a production line at a large rice processing plant. This line has a daily processing capacity of 300 tons and mainly processes three varieties of rice: indica rice, japonica rice, and glutinous rice. The detection system is deployed in the middle section of the conveying pipeline of elevator No. 17, with a detection area length of 2 meters, a pipeline diameter of 400 mm, and a rice flow rate controlled at 2.5 tons per minute. During the experiment, test data were collected from three rice varieties. Each variety was tested continuously for 8 hours, and the valid sample data are shown in Table 1.

[0080] The comparison results of process parameters before and after optimization are shown in Table 2:

[0081] The effect of optimizing the broken rice rate is as follows: Figure 3 As shown, through intelligent process parameter optimization, the broken rice rate decreased from an average of 6.9% to 4.3%, approaching the target value of 4.0%. The optimized broken rice rate showed more stable changes, with the standard deviation decreasing from 0.31 to 0.24, indicating a significant improvement in the stability of the production process. Simultaneously, the output remained relatively stable, and production efficiency was not affected by the optimization. The embodiments of the present invention have been described above. However, the embodiments are not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make more equivalent embodiments under the guidance of the present embodiments, and all of them are within the protection scope of the present embodiments.

Claims

1. An online detection method for the broken rice rate of grains based on image analysis, characterized in that, Includes the following steps: Grain particle flow images are acquired by a multi-angle camera array and a stroboscopic light source system. Multi-angle light source configuration and polarization filtering technology are used to eliminate reflective interference and output a particle flow image sequence. Based on granular image sequences, a reinforcement learning agent based on a deep Q-network is constructed. The state space includes environmental parameters and image quality indicators, and the action space includes continuous parameters for contrast adjustment, denoising intensity, and color correction. The reinforcement learning agent selects the optimal combination of image enhancement parameters and outputs the enhanced image. Particle detection and segmentation are performed on the enhanced image. A lightweight YOLOv8 network is used for fast particle detection and localization. The particle regions located in the coarse detection stage are accurately segmented. An improved MaskR-CNN network is used to output particle region and contour information. Based on granular region and contour information, geometric, texture, color and shape features are extracted; multimodal features are fused and a multi-branch feature fusion network is designed, with four branches processing geometric, texture, color and shape features respectively. The correlation between different modal features is calculated through a cross attention mechanism, and a deep fusion feature vector is output. The deep fusion feature vector is input into the ensemble classifier to construct three different types of classifiers: a lightweight convolutional neural network, a Transformer classifier, and an XGBoost gradient boosting tree. The weights of each classifier are dynamically adjusted according to the feature distribution and classification difficulty of the input samples. A confidence-based weight allocation method is adopted, with classifiers with higher confidence being assigned greater weights. A weighted voting method is used to make the final classification decision and obtain the particle classification result. Based on the results of particle classification, the real-time broken rice rate is calculated, a digital twin model of the rice milling process is established, and the complex relationship between the process parameters of rice milling pressure, rotation speed, and feed rate and the broken rice rate is simulated. The model predictive control algorithm is used to predict the future trend of broken rice rate. According to the output of the predictive control algorithm, the pressure and rotation speed parameters of the rice milling machine are automatically adjusted; the working parameters of the rice milling machine are optimized and adjusted in real time. An incremental learning database is established to collect historical data for testing. Knowledge distillation technology is used to update the model, enabling continuous learning and knowledge updates. The incremental learning database is constructed with a hierarchical storage structure and indexed by variety, time, and process parameters. Knowledge distillation technology is used to transfer knowledge from the complex teacher model to the lightweight student model. The knowledge distillation process includes two parts: hard label loss and soft label loss. An elastic weight consolidation algorithm is used to prevent catastrophic forgetting. Fisher information matrix of parameter importance is calculated, and stronger regularization constraints are applied to important parameters. An experience playback mechanism is established to interleave historical data during the training of new data to maintain the memory of old knowledge.

2. The online detection method for grain broken rice rate based on image analysis according to claim 1, characterized in that, The step of acquiring images of grain particle flow also includes using laser line scanning imaging technology: An 808 nm near-infrared laser is used as the light source, combined with a CMOS linear array sensor for imaging. The laser has a power of 50 watts, a beam width of 2 millimeters, and a scanning frequency of 2 kilohertz. The laser imaging system uses the triangulation principle to simultaneously acquire the geometric shape and surface texture information of grain particles.

3. The online detection method for grain broken rice rate based on image analysis according to claim 1, characterized in that, The multi-angle camera array and strobe light source system includes: Three sets of linear array cameras are installed in the middle section of the conveying pipeline, including a main camera that points vertically downward from the top and auxiliary cameras that are tilted at 45 degrees on both sides, forming a triangular imaging geometry. The main camera has a resolution of 4096×2048 pixels, the auxiliary camera has a resolution of 2048×1024 pixels, and the cameras are synchronized by hardware, with the synchronization accuracy controlled within 1 microsecond. It is equipped with a ring-shaped strobe LED light source array, which contains 12 independently controlled light source units, each with a power of 200W and a color temperature of 6500K. The light source frequency is precisely synchronized with the camera frame rate, and the strobe frequency is 1000Hz.

4. The online detection method for grain broken rice rate based on image analysis according to claim 1, characterized in that, The reinforcement learning agent includes: Deploy a multi-source environmental sensor network, including illuminance sensors, dust concentration detectors, vibration accelerometers, and temperature and humidity sensors; The agent selects the optimal combination of image enhancement parameters based on the current state, including gamma correction coefficients, Gaussian filter kernel size, bilateral filter parameters, and histogram equalization intensity.

5. The online detection method for grain broken rice rate based on image analysis according to claim 1, characterized in that, The particle detection and segmentation of the enhanced image includes: The network input size is 640×640 pixels, the backbone network uses CSPDarknet53, and the neck network uses PANet structure; The backbone network uses ResNet-101, and the feature pyramid network adopts the FPN structure; Temporal information constraints are used to eliminate false detections caused by changes in illumination and particle motion. A particle tracking mechanism is established, Kalman filtering is used to predict particle trajectories, and the Hungarian algorithm is combined for data association.

6. The online detection method for grain broken rice rate based on image analysis according to claim 1, characterized in that, The fused multimodal features include: For each segmented particle, calculate the geometric feature parameters, extract the particle outline, and simplify the outline using the Douglas-Peucker algorithm; calculate the basic geometric features such as area, perimeter, length and width of the minimum bounding rectangle, centroid coordinates, and principal axis direction angle. Texture features are calculated based on the gray-level co-occurrence matrix, including contrast, correlation, energy, homogeneity, and entropy statistics, with parameter combinations of four directions and three distances set. Convert the RGB image to HSV and Lab color spaces, calculate the mean, standard deviation, skewness, and kurtosis statistical characteristics of each channel, and calculate the color histogram characteristics.

7. The online detection method for broken rice rate of grains based on image analysis according to claim 1, characterized in that, The ensemble classifier includes: The CNN classifier uses the MobileNetV3 architecture and contains 13 inverse residual blocks; The voting weight of each classifier is determined by its dynamic weight and historical accuracy, and broken rice is further subdivided into three grades based on grain length.

8. The online detection method for broken rice rate of grains based on image analysis according to claim 1, characterized in that, The real-time optimization and adjustment of the rice milling machine's operating parameters includes: The model uses a deep neural network structure, which includes an input layer, three hidden layers, and an output layer; Within a sliding time window, the number of various types of particles is counted, the real-time broken rice rate is calculated, and a weighted average method is used to process historical data, with recent data having a higher weight. The rice milling process parameters were optimized, with the prediction domain set to 10 sampling periods and the control domain set to 5 sampling periods.

9. The online detection method for broken rice rate of grains based on image analysis according to claim 1, characterized in that, The continuous learning and knowledge updating includes: Each batch of new samples is processed at a time, with a batch size of 32; the teacher model is a complete ensemble classifier, and the student model is a single lightweight network; the pruning ratio is 30%, and the quantization precision is an 8-bit integer; the model update frequency is adaptively adjusted according to changes in detection precision, and a model update is triggered when the precision drops by more than 5%.

10. An online grain breakage rate detection system based on image analysis, used to execute the online grain breakage rate detection method based on image analysis as described in any one of claims 1-9, characterized in that, include: The image acquisition module is used to acquire images of grain particle flow through a multi-angle camera array and a strobe light source system. It uses a multi-angle light source configuration and polarization filtering technology to eliminate reflective interference and outputs a sequence of particle flow images. The image enhancement module is used to select the optimal combination of image enhancement parameters based on the granular flow image sequence and output the enhanced image. The particle detection module is used to detect and segment particles in the enhanced image, and output particle region and contour information. The feature extraction module is used to extract geometric, texture, color, and shape features based on granular region and contour information; and to fuse these features. Multimodal features, outputting a deep fusion feature vector; The classification and recognition module is used to input the deep fusion feature vector into the ensemble classifier to obtain the particle classification result; The control optimization module is used to calculate the real-time broken rice rate based on the particle classification results and to predict the rice milling trend using a predictive model. Based on predicted rice milling trends, the operating parameters of the rice milling machine are optimized and adjusted in real time. The learning and updating module is used to build an incremental learning database to collect historical data and update the model using knowledge distillation technology, thereby achieving continuous learning and knowledge updating.

Images