Wheat imperfect grain recognition method and wheat imperfect grain recognition device

Abstract

The invention provides a wheat imperfect grain recognition method and a wheat imperfect grain recognition device. The method comprises the following steps: enabling wheat in a wheat sample to be detected to drop one by one; in the falling process of each wheat grain, at least two cameras are adopted to shoot each wheat grain so as to at least obtain images on the two sides of each wheat grain, and at least two images to be recognized are obtained; identifying the at least two to-be-identified images based on a constructed deep learning network model suitable for wheat imperfect grain images to obtain the grain type of each grain of wheat; and sorting each wheat grain to a corresponding collection area based on the grain type. According to the scheme, the recognition speed and the recognition precision of the wheat imperfect grains can be improved, the required calculation amount is small, and the method can be deployed in a resource-limited production environment.

Description

Technical Field

[0001] The present invention relates to a technology for identifying imperfect wheat grains, and in particular to a method and a device for identifying imperfect wheat grains. Background Art

[0002] As a key food source, wheat plays a vital role in human production and daily life. Wheat imperfection detection aims to improve grain quality, remove impurities and inferior grains, and ensure grain storage and food safety. Improving the accuracy and efficiency of imperfect wheat kernel identification can help grain procurement, storage, and food processing companies achieve precise quality control, thereby increasing their economic value.

[0003] Currently, wheat defect detection relies primarily on manual screening and visual inspection, which is inefficient and highly subjective, making it difficult to meet the modern grain industry's demand for standardized testing. Traditional automated equipment relies on physical property analysis, but lacks accuracy in identifying complex damaged and diseased kernels. Summary of the Invention

[0004] In view of the above problems, the present invention is proposed to provide a method and device for identifying imperfect wheat grains that overcome the above problems or at least partially solve the above problems. It can improve the recognition speed and recognition accuracy of imperfect wheat grains, requires little computational effort, and can be deployed in a resource-constrained production environment.

[0005] Specifically, the present invention provides a method for identifying imperfect wheat grains, which comprises:

[0006] causing wheat grains in the wheat sample to be tested to fall one by one;

[0007] During the falling process of each grain of wheat, at least two cameras are used to photograph each grain of wheat, so as to obtain images of at least two sides of each grain of wheat, thereby obtaining at least two images to be identified;

[0008] Identify at least two of the images to be identified based on a deep learning network model that has been constructed and is suitable for images of imperfect wheat grains, to obtain the type of each grain of wheat;

[0009] Each grain of wheat is sorted into a corresponding collection area based on the grain type.

[0010] Optionally, the constructed deep learning network model suitable for wheat imperfect grain images includes a first deep learning network model and a second deep learning network model, and the first deep learning network model identifies more types of grains than the second deep learning network model.

[0011] The step of identifying at least two images to be identified based on the deep learning network model that has been constructed and is applicable to the image of imperfect wheat grains to obtain the type of each grain of wheat includes:

[0012] Identify at least two of the to-be-identified images based on the first deep learning network model to obtain the grain type of each grain of wheat, until a first preset number of grain types of wheat are obtained;

[0013] Determining the second deep learning network model according to a first preset number of wheat grain types;

[0014] Based on the second deep learning network model, each of the remaining wheat images to be identified is identified to obtain the particle type of each remaining wheat grain.

[0015] Optionally, the construction of the first deep learning network model and the construction of the second deep learning network model both include:

[0016] Build the MobileNetV3 basic network;

[0017] Add a coordinate attention module before the neck module in the MobileNetV3 base network;

[0018] Use a dual-branch channel attention module to replace the SE attention module in the MobileNetV3 base network;

[0019] Use the multi-scale feature fusion module to replace the feature fusion module in the MobileNetV3 basic network to obtain the first deep learning network model and the second deep learning network model;

[0020] The first deep learning network model and the second deep learning network model are trained using a picture set of wheat grain types to obtain the trained first deep learning network model and the second deep learning network model.

[0021] Optionally, the identifying of at least two images to be identified based on a deep learning network model that has been constructed and is suitable for images of imperfect wheat grains to obtain the type of each grain of wheat includes:

[0022] identifying each image to be identified based on a deep learning network model that has been constructed and is suitable for images of imperfect wheat grains, thereby obtaining at least two initial grain types for each grain of wheat; the initial grain types include perfect grains and imperfect grains, and the imperfect grains include insect-damaged grains, diseased grains, heat-damaged grains, broken grains, sprouted grains, moldy grains, and / or shriveled grains;

[0023] determining whether at least two of the initial particles are of the same type;

[0024] If so, the initial grain type is used as the grain type of the wheat;

[0025] If not, the initial grain type that is the imperfect grain and has the highest confidence among the at least two initial grain types is used as the grain type of the wheat grain.

[0026] Optionally, the method for identifying imperfect wheat grains further comprises:

[0027] The lighting device located on the upper side of the falling path of the wheat is controlled to emit light, so as to provide lighting when the camera takes a picture of each grain of wheat.

[0028] Optionally, the method for identifying imperfect wheat grains further comprises:

[0029] The brightness and illumination angle of the lighting device are determined according to a first preset number of wheat grain types, and the shooting angle of the camera is controlled.

[0030] Optionally, the method for identifying imperfect wheat grains further comprises:

[0031] obtaining the quantity and weight of wheat in each of the collection areas;

[0032] Based on the quantity and weight of the wheat in each of the collecting areas and the quantity and weight of the wheat in the wheat sample to be tested, the quantity ratio and mass ratio of the wheat in each of the collecting areas are obtained.

[0033] The present invention also provides a device for identifying imperfect wheat grains, which includes a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps of any of the above-mentioned imperfect wheat grain identification methods.

[0034] Optionally, the device for identifying imperfect wheat grains further comprises:

[0035] a feeding device configured to receive a wheat sample to be tested and to cause the wheat in the wheat sample to be tested to fall down grain by grain;

[0036] a conveyor belt, horizontally arranged at the lower side of the feeding device, configured to receive the wheat from the feeding device and convey the wheat so that the wheat falls from one end of the conveyor belt; a paddle is provided on the conveyor belt;

[0037] A lighting device is provided on the upper side or the lower side of one end of the conveyor belt;

[0038] At least two of the cameras are disposed on the lower side of one end of the conveyor belt and are evenly distributed along the circumference of the falling path; and

[0039] A separation device is configured to receive the wheat from the conveyor belt and move the wheat to a corresponding collection area.

[0040] Optionally, the separation device includes:

[0041] A collection box having two collection areas with openings facing upward;

[0042] A partition is horizontally arranged on the upper side of the two collecting areas; and the partition is rotatably arranged so that the partition is inclined toward one collecting area or toward the other collecting area, thereby allowing the wheat on the partition to enter the corresponding collecting area.

[0043] In the wheat imperfect grain identification method and wheat imperfect grain identification device of the present invention, dual-camera collaborative imaging is used to significantly reduce the risks of false detection and missed detection caused by single-view occlusion; the designed lightweight model is superior to mainstream lightweight models in recognition accuracy and parameter quantity, and can be deployed on embedded devices or mobile terminals to meet the needs of efficient detection in low-resource scenarios; it is suitable for grain storage, processing enterprises and quality inspection agencies, and can realize automated statistics of the quantity and quality of imperfect grains, providing precise inspection technical support for grain procurement, storage, transportation and processing.

[0044] Based on the following detailed description of specific embodiments of the present invention in conjunction with the accompanying drawings, those skilled in the art will become more aware of the above and other objects, advantages and features of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Hereinafter, some specific embodiments of the present invention will be described in detail in an illustrative and non-limiting manner with reference to the accompanying drawings. The same reference numerals in the accompanying drawings indicate the same or similar components or parts. It should be understood by those skilled in the art that these drawings are not necessarily drawn to scale. In the accompanying drawings:

[0046] Figure 1 is a schematic structural diagram of a device for identifying imperfect wheat grains according to one embodiment of the present invention;

[0047] Figure 2 is a schematic structural diagram of a separation device in a device for identifying imperfect wheat grains according to one embodiment of the present invention;

[0048] Figure 3 1 is a schematic structural diagram of a feeding device in a device for identifying imperfect wheat grains according to one embodiment of the present invention;

[0049] Figure 4 is a schematic flow chart of a method for identifying imperfect wheat kernels according to one embodiment of the present invention;

[0050] Figure 5is a partial flow chart of a method for identifying imperfect wheat grains according to one embodiment of the present invention;

[0051] Figure 6 is a schematic diagram of a deep learning network model in a method for identifying imperfect wheat grains according to an embodiment of the present invention;

[0052] Figure 7 yes Figure 6 Schematic diagram of the coordinate attention module in the deep learning network model shown;

[0053] Figure 8 yes Figure 6 Schematic diagram of the dual-branch channel attention module in the deep learning network model shown;

[0054] Figure 9 yes Figure 6 Schematic diagram of the multi-scale feature fusion module in the deep learning network model shown;

[0055] Figure 10 for Figure 6 Schematic diagram of wheat grain images used in training the deep learning network model;

[0056] Figure 11 for Figure 6 Schematic diagram of image enhancement changes during deep learning network model training. DETAILED DESCRIPTION

[0057] Refer to the following Figures 1 to 11 The following describes the scheme of the method and device for identifying imperfect wheat grains provided by the embodiments of the present invention.

[0058] Figure 1 FIG. 1 is a schematic structural diagram of a device for identifying imperfect wheat grains according to an embodiment of the present invention. Figure 1 As shown, an embodiment of the present invention provides a device for identifying imperfect wheat grains, which includes a feeding device 11, a conveyor belt 12, a lighting device 13, at least two cameras 14 and a separation device.

[0059] The feeder 11 is configured to receive a wheat sample to be tested and allow the wheat grains in the sample to fall one by one. When in use, the wheat sample to be tested is placed in the feeder 11, and the wheat grains fall out of the feeder 11 one by one. The wheat sample to be tested typically contains 50 grams of wheat.

[0060] The conveyor belt 12 is horizontally arranged at the lower side of the feeding device 11, and is configured to receive the wheat from the feeding device 11 and convey the wheat so that the wheat falls from one end of the conveyor belt 12. The conveyor belt 12 is provided with a paddle.

[0061] The lighting device 13 is positioned above or below one end of the conveyor belt 12. The lighting device 13 can be an annular light-emitting device, with the wheat's falling path coaxially aligned with the annular light-emitting device. The annular light-emitting device can be an LED light source. In alternative embodiments of the present invention, other types of lighting devices 13, such as LED bulbs, can also be employed.

[0062] At least two cameras 14 are positioned below one end of the conveyor belt 12 and evenly spaced along the circumference of the falling path. Triggering the dual cameras 14 during the grain's fall allows for simultaneous capture of images of both the front and back sides of the grain. This prevents missed detections caused by occlusions in the single-viewing angle of a single camera 14. For example, if a wormhole is located on the side of the grain facing away from the single camera 14, it could lead to incorrect recognition and, consequently, missed detection.

[0063] The separating device is configured to receive the wheat from the conveyor belt 12 and move the wheat to a corresponding collection area.

[0064] When the imperfect wheat grain identification device of the embodiment of the present invention is working, the feeding device 11 allows the wheat grains in the received wheat sample to be tested to fall onto the conveyor belt 12 one by one, and the conveyor belt 12 drives the wheat grains to move to one end of the conveyor belt 12. The wheat grains move downward from one end of the conveyor belt 12 under the action of gravity. The lighting device 13 is always in an on state, used to illuminate the surroundings of the wheat grains. When the wheat grains fall to a certain position, multiple cameras 14 start working at the same time to obtain photos of at least two sides of the wheat grains, that is, the images to be identified. The processor of the imperfect wheat grain identification device obtains the particle type of each grain of wheat based on at least two images to be identified and the built-in deep learning network model. Based on the particle type, the separation device sorts each grain of wheat into the corresponding collection area.

[0065] The device for identifying imperfect wheat grains in an embodiment of the present invention: through the collaborative imaging of the dual cameras 14, the risk of false detection and missed detection caused by single-view occlusion is significantly reduced, ensuring that no wheat grain defects are missed, and the accuracy of identifying grain types can be improved.

[0066] In some embodiments of the present invention, Figure 1 As shown, the separation device includes a collecting box and a partition 15. The collecting box has two collecting areas 16 with openings facing upward. The partition 15 is horizontally arranged on the upper side of the two collecting areas 16. And the partition 15 is rotatably arranged so that the partition 15 is tilted toward one collecting area 16 or tilted toward the other collecting area 16, thereby allowing the wheat on the partition 15 to enter the corresponding collecting area 16. In an embodiment of the present invention, there are two collecting areas 16, namely, an imperfect grain collection area 16 and a perfect grain collection area 16. The perfect grains identified by the deep learning network model will be moved to the perfect grain collection area 16, and the other wheat grains will be collected in the imperfect grain collection area 16.

[0067] In some preferred embodiments of the present invention, Figure 2 As shown, the separation device includes a collecting box and a receiving separator. The collecting box has a plurality of collecting areas 16 with openings facing upward, and the collecting areas 16 are evenly distributed along the circumference of the collecting box. The receiving separator includes a receiving groove 17 and a guide cylinder 18. The receiving groove 17 opens upward to receive the wheat grains, and an opening is provided at the bottom. The guide cylinder 18 is arranged at an angle, and the upper end of the guide cylinder 18 is connected to the bottom opening of the receiving groove 17 to receive the wheat grains. The upper end of the guide cylinder 18 rotates around the vertical axis and is connected to the lower side of the receiving groove 17, so that the lower end of the guide cylinder 18 rotates to the upper side of the corresponding collecting area 16, and then the received wheat grains are moved to the corresponding collecting area 16. In this embodiment, each type of wheat can be moved to its respective collection area 16.

[0068] In some embodiments of the present invention, Figure 3 As shown, to ensure that the wheat in the wheat sample to be tested falls down one grain at a time, the feeding device 11 includes a feeding channel 21. The feeding channel 21 is larger at the top and smaller at the bottom, with the upper end of the channel larger than the lower end. A screw conveyor 22 is located within the feeding channel 21. After the wheat enters the feeding channel, the screw conveyor 22 rotates to drive the wheat upward, while the wheat at the bottom moves downward under its own weight. The opening between the lower end of the screw conveyor and the wall of the feeding channel is only large enough to allow one wheat grain to fall, not two. In other words, the opening between the lower end of the screw conveyor and the wall of the feeding channel is larger than one wheat grain and smaller than two wheat grains, and can be 1.1 times the size of one wheat grain. In this embodiment of the present invention, the rotation of the screw conveyor drives the wheat upward rather than downward, preventing the wheat from congregating at the lower end under the drive of the screw conveyor and being unable to flow out. The screw conveyor drives the wheat upward, allowing the wheat at the bottom to disperse upward, while the wheat at the bottom moves freely downward under its own weight. The spiral channel formed between the spiral conveying device and the wall of the feed channel gradually becomes larger from bottom to top, ensuring the smooth transportation of wheat.

[0069] In some embodiments of the present invention, the feeding device 11 is a funnel, and the feeding channel is the neck of the funnel. The diameter of the funnel neck is generally no larger than the length of the single grain of wheat, so that only one grain of wheat can pass through the bottom of the funnel neck at a time.

[0070] In some embodiments of the present invention, the paddles are strip-like or comb-like protrusions evenly arranged on the conveyor belt, made of a flexible material. During the conveyor's operation, they physically paddle and space the wheat kernels at the bottom of the feed channel, pushing out the wheat kernels stacked in the feed device 11 one by one. The pushed-out wheat kernels are then arranged and transported at a fixed spacing, and finally delivered to one end of the conveyor belt where they fall. The conveyor belt speed is adjustable. Furthermore, the lower end of the feed channel is made of soft plastic.

[0071] In some embodiments of the present invention, the imperfect wheat kernel identification device further includes a wheat kernel detection device configured to detect the presence of wheat kernels at one end of the conveyor belt 12. After the wheat kernel detection device detects the presence of wheat kernels, the camera 14 is controlled to operate according to a preset time. Specifically, after the wheat kernel detection device detects the presence of wheat kernels, the camera 14 is activated after a preset time. After the preset time, the wheat kernels will fall to a position corresponding to the camera 14, allowing the camera 14 to capture images.

[0072] In some alternative embodiments of the present invention, the imperfect wheat kernel identification device further includes a wheat kernel position detection device configured to detect the falling position of the wheat. When the wheat kernel position detection device detects wheat at the position corresponding to camera 14, it controls camera 14 to operate. The wheat kernel position detection device is an infrared photoelectric sensor.

[0073] In some embodiments of the present invention, in order to improve the shooting effect, the shooting angle of the camera 14 is tilted relative to the horizontal plane, that is, the shooting line of sight of the camera 14 is directed diagonally upward or diagonally downward. The angle between the shooting line of sight of the camera 14 and the horizontal plane is within 10° to prevent the angle from being too large and failing to obtain a complete image of the side of the wheat grain. Furthermore, since the lighting device 13 is arranged on the upper side of the camera 14, the shooting line of sight of the camera 14 is directed diagonally downward. The lens of the camera 14 has a certain tilt angle relative to the horizontal plane, so that the camera 14 will not become the background of the shooting. The camera 14 is preferably arranged at 13cm to 17cm where the wheat is freely falling, preferably at 15cm, that is, the distance between the camera 14 and one end of the conveyor belt 12 is 13cm to 17cm. At this height, the movement of the wheat is relatively stable, and the falling speed is not very fast, which is conducive to the shooting of the camera 14.

[0074] In some embodiments of the present invention, the wheat imperfect grain identification system further includes a weighing device 19 and a display device. The weighing device is configured to detect the weight of the wheat in each collection area 16. Specifically, Figure 2 As shown, each collection area 16 is a collection bucket, and the weighing device 19 is a pressure sensor installed on the bottom of each collection bucket to determine the mass of the wheat in the collection bucket based on the pressure change. The display device is used to display wheat-related information such as the identification result (grain type), quantity percentage, and mass percentage.

[0075] Figure 4 FIG. 1 is a schematic flow chart of a method for identifying imperfect wheat grains according to an embodiment of the present invention. Figure 4 As shown, an embodiment of the present invention further provides a method for identifying imperfect wheat grains, which includes:

[0076] Step S100 , making the wheat in the wheat sample to be tested fall down grain by grain.

[0077] Step S200 , using at least two cameras 14 to photograph each grain of wheat as it falls, so as to obtain images of at least two sides of each grain of wheat, thereby obtaining at least two images to be identified.

[0078] Step S300 identifies at least two images to be identified based on a deep learning network model constructed for images of imperfect wheat kernels, and determines the kernel type of each kernel. Kernel types may include perfect kernels, insect-damaged kernels, diseased kernels, heat-damaged kernels, broken kernels, sprouted kernels, moldy kernels, and / or shriveled kernels. The identification type can be determined based on national grain and oil inspection standards, or based on training samples and actual user needs. Except for perfect kernels, all other kernel types are considered imperfect kernels.

[0079] In step S400 , each grain of wheat is sorted into a corresponding collection area 16 based on the grain type.

[0080] In this embodiment of the present invention, the collaborative imaging of the dual cameras 14 significantly reduces the risk of false detection and missed detection caused by single-view occlusion, thereby improving the accuracy of grain type recognition. By recognizing at least two images to be identified based on a deep learning network model developed for images of imperfect wheat grains, automated recognition with high accuracy can be achieved.

[0081] In some embodiments of the present invention, a deep learning network model constructed for images of imperfect wheat grains includes a first deep learning network model and a second deep learning network model. The first deep learning network model identifies more types of grains than the second deep learning network model. Because the first deep learning network model identifies more types of grains than the second deep learning network model, the computational complexity when using the first deep learning network model is greater than that when using the second deep learning network model. Both the first deep learning network model and the second deep learning network model are capable of identifying perfect grains.

[0082] In the embodiment of the present invention, the types of grains in the same batch of wheat are relatively fixed and generally do not contain all types of grains. In addition, a certain type of grain type accounts for a significantly larger proportion and is the main grain type in the batch of wheat. Therefore, the grain type with a significantly larger proportion is preferentially screened out to reduce the amount of processor computation and improve processing efficiency. Specifically, Figure 5 As shown, the above step S300, which identifies at least two images to be identified based on the deep learning network model that has been constructed for the imperfect wheat grain image, to obtain the grain type of each wheat grain, includes:

[0083] Step S310: Identify at least two images to be identified based on the first deep learning network model to obtain the particle type of each grain of wheat, until a first preset number of wheat particle types are obtained.

[0084] Step S320: Determine a second deep learning network model based on a first preset number of wheat grain types.

[0085] Step S330: Identify each remaining wheat image based on the second deep learning network model to obtain the grain type of each remaining wheat grain.

[0086] In this embodiment of the present invention, multiple second deep learning network modules can be used to correspond to different particle types or combinations of particle types. The first deep learning network model is used to identify one or more primary particle types, and then a corresponding second deep learning network model with a relatively low computational load is selected to significantly improve the efficiency of particle type recognition.

[0087] Furthermore, in some embodiments of the present invention, wheat kernels whose types cannot be identified by the second deep network learning module can be stored separately in corresponding collection area 16. After all the wheat in the wheat sample to be tested has been identified, the first deep learning network model is used to re-identify these wheat kernels. Using both the first and second deep network learning modules can reduce the computational complexity of recognition and significantly improve recognition efficiency.

[0088] In some alternative embodiments of the present invention, only the first deep learning network model is used to identify wheat grain types. Compared with the previous embodiment, this recognition strategy will increase the amount of calculation, but the identified grain types are more comprehensive.

[0089] In some embodiments of the present invention, the first deep learning network model and the second deep learning network model are both lightweight recognition networks, referred to as LIWR models. Specifically, the construction of the first deep learning network model and the second deep learning network model includes:

[0090] Build the MobileNetV3 basic network.

[0091] Add a coordinate attention module before the neck module in the MobileNetV3 base network.

[0092] The SE attention module in the MobileNetV3 base network is replaced by a dual-branch channel attention module.

[0093] The multi-scale feature fusion module is used to replace the feature fusion module in the MobileNetV3 basic network to obtain the first deep learning network model and the second deep learning network model.

[0094] The first deep learning network model and the second deep learning network model are trained using a picture set of wheat grain types to obtain the trained first deep learning network model and the second deep learning network model.

[0095] In the embodiment of the present invention, Figure 6 As shown in the figure, the first deep learning network model and the second deep learning network model are both lightweight recognition networks, called LIWR models, and both have basic network structures such as convolutional layers, activation layers, pooling layers, fully connected layers, and outputs. The coordinate attention (CA) module combines image position information with spatial information to enhance the model's focus on important information and improve its ability to distinguish different types of wheat grains. The dual-branch channel attention (E2CA) module is used in the residual block to effectively learn the dependencies between channels while reducing the number of model parameters, highlighting important features through precise attention guidance. The multi-scale feature fusion (MFF) module enables the model to effectively obtain features of different receptive fields and different scales by combining operations such as depthwise convolution and depthwise separable convolution in parallel.

[0096] In an embodiment of the present invention, the first deep learning network model and the second deep learning network model have low computational complexity compared to existing models such as ResNet and VGG, are easily deployed on embedded devices or mobile terminals, and have little performance degradation in scenarios where computing and other resources are limited. They have a strong ability to distinguish imperfect particles with high similarity (such as worm-eaten particles and broken particles), and therefore have high recognition accuracy. The first deep learning network model and the second deep learning network model of the embodiment of the present invention adopt a lightweight design and incorporate multiple attention mechanisms to achieve a balance between parameter quantity and accuracy.

[0097] In some embodiments of the present invention, a coordinate attention (CA) module is added after the first convolutional layer of the network. The module first performs global pooling of the input feature map in the horizontal and vertical directions to capture spatial position information. The two directional encoding results are then concatenated and intermediate features are generated through 1×1 convolution and nonlinear activation. Finally, the intermediate features are split into horizontal and vertical attention weights, which are then element-wise multiplied with the original feature map after Sigmoid activation to output position-aware weighted features. This structure enhances the model's focus on important information by explicitly modeling the relationship between position and channel, effectively improving the model's sensitivity to local wheat defects while maintaining a lightweight architecture.

[0098] Specifically, if Figure 7As shown in the figure, a coordinate attention (CA) module is added before the neck module (also called the neck network). This module combines positional and spatial information and weights image information, enabling the model to prioritize important feature channels in the initial stages. This enhances feature extraction, reduces image feature loss, and improves the ability to distinguish different types of wheat grains. The coordinate attention (CA) module applies horizontal and vertical pooling operations to each channel of the input feature map using convolutional kernels of size (H, 1) and (1, W) to extract basic spatial details. The formula is as follows:

[0099] and

[0100] Among them, H and W represent the height and width of the input feature X respectively, Represents the pooling operation result of the c channel in the height direction, Represents the pooling operation result of the c channel in the width direction.

[0101] Subsequently, the outputs of the above processes are concatenated, and an intermediate feature map is generated through convolution transformation, and then an activation operation is performed. The formula is as follows:

[0102] f=δ(F1([y h ,y w ])), where [·,·] represents the concatenation operation, F1 represents the convolution kernel used for convolution transformation, δ is the nonlinear activation function, and f is the intermediate feature map.

[0103] Then, the intermediate feature map is split horizontally and vertically to obtain two separate feature tensors. A 1×1 convolution transformation is applied to each separated feature tensor to produce the corresponding attention weight. The formula is: g h =σ(F h (f h )) and g w =σ(F w (f w )). Among them, F h and F w is a convolution transformation using a 1×1 convolution kernel, and σ is a sigmoid activation function. Finally, the feature Figure X Multiply by g h and g w To obtain the final result Y c (i,j), the specific formula is

[0104] In some embodiments of the present invention, the dual-branch channel attention (E2CA) module utilizes two ECA branches in parallel, extracting channel features through global average pooling and 1D convolution, respectively. Sigmoid activation is applied to each output to obtain channel attention weights. The dual-branch weights are then fused and multiplied with the original feature map channel by channel to enhance key features. This module optimizes the channel attention mechanism through a dual-branch structure. Compared to the channel attention SE module, this design reduces the number of fully connected layers and utilizes 1D convolution for cross-channel interaction, reducing the number of parameters by 50% while improving the ability to distinguish similar grain features.

[0105] Specifically, if Figure 8 As shown in Figure 2, the SE module is replaced with a bidirectional ECA module at the bottleneck. The feature map is processed by two ECA branches to more effectively learn channel dependencies. Compared with the SE module, the use of two fully connected layers reduces the number of parameters and computational overhead. Figure 5 In the

[15] , each ECA branch first performs global average pooling on the input feature map, then performs a one-dimensional convolution operation with a kernel size of k, and then uses the Sigmoid function to calculate the channel weights and multiply the original feature map by these weights to generate the output result.

[0106] Multi-scale Feature Fusion (MFF) module: This module adopts a parallel hybrid structure to achieve efficient feature extraction. It uses depthwise convolution and depthwise separable convolution in parallel to capture fine-grained features of different receptive fields. It then retains the original features through shortcut connections and weightedly fuses them with multi-scale output features. Depthwise separable convolution reduces the amount of computation and significantly reduces the number of parameters compared to traditional multi-scale modules.

[0107] Specifically, if Figure 9 As shown in the figure, the Multi-Scale Feature Fusion (MFF) module replaces the traditional model that only uses shortcut connections to reuse features from shallow to deep layers. The Multi-Scale Feature Fusion (MFF) module includes depthwise convolution, depthwise separable convolution, and shortcut connections. This enables the model to effectively acquire deep features at different receptive field scales, enhancing the ability to recognize wheat grains while maintaining the model's lightweight.

[0108] In some embodiments of the present invention, a first deep learning network model and a second deep learning network model are trained using a set of pictures of wheat grain types. Data enhancement techniques such as horizontal flipping, brightness adjustment, and Gaussian noise injection can be used to increase the diversity of training samples, thereby improving the generalization ability and robustness of the model. Furthermore, a cosine annealing strategy can be used to dynamically adjust the learning rate, making it easier for the model to reach global optimality. Specifically, the model training data set can use images captured and annotated by the camera 14 of the present invention, or use public data sets such as GrainSpace, such as Figure 10In order to enhance the robustness of the model to different wheat varieties, lighting, noise and other conditions, data enhancement techniques such as horizontal flipping, brightness adjustment and Gaussian noise injection can be used to increase the diversity of training samples. Figure 11 As shown. During model training, an optimization strategy for dynamically adjusting the learning rate is employed, employing a cosine annealing strategy for learning rate adjustment. Using a large learning rate in the early stages facilitates rapid convergence and escape from the local optimum, while a small learning rate is used later to facilitate approaching the global optimum. In a deep learning network model, training images can be acquired through the camera 14 in the imperfect wheat grain recognition device. In this case, the imperfect wheat grain recognition device is not used for recognition, but only for acquiring training images, resulting in a relatively efficient image acquisition.

[0109] In order to evaluate the performance of the wheat imperfect grain recognition model (deep learning network model) constructed in this embodiment, seven typical imperfect grain samples were extracted from the large-scale grain appearance detection dataset Grainspace. 1000 images were randomly selected from each category, and the dataset was split at a ratio of 8:1:1 to obtain a training set of 5600 images, a validation set of 700 images, and a test set of 700 images. Some examples of wheat grain images from GrainSpace are as follows: Figure 10 shown.

[0110] The training process for the deep learning network model is as follows: The training and validation set images are fed into the constructed deep learning network model. The model is trained for 100 iterations, using Adam as the optimizer, and a batch size of 32. During training, a cosine annealing strategy is used to adjust the learning rate, with an initial learning rate of 0.01. As training progresses, the learning rate is gradually reduced using a cosine function curve. This allows the model to converge faster and avoid local optima at a higher learning rate in the early stages of training, while approaching the global optimum at a lower learning rate later in the training process. After training is complete, the model weight parameters are saved.

[0111] The performance of the deep learning network model was evaluated on the test set, using accuracy, precision, recall, and F1-score to measure the performance of the model, and model parameters and the number of floating-point operations (FLOPs) to measure the model complexity. A comprehensive comparative analysis was conducted on the deep learning network model proposed in this invention with classic convolutional neural networks and lightweight models, including VGG-16, ResNet-50, GoogleLeNet, ShuffleNet-V2, EfficientNet-B0, and MobileNetV3-Small. The comparison results of the present invention with the current advanced image recognition models on the sampled GrainSpace dataset are shown in Table 1.

[0112] Table 1

[0113]

[0114] Experimental examples demonstrate that the proposed deep learning network model, LIWR, exhibits superior performance while maintaining low computational complexity, outperforming existing models across almost every evaluation metric. It surpasses the baseline MobileNetV3 Small in accuracy and F1 score by 6.67% and 7.18%, respectively.

[0115] In some embodiments of the present invention, at least two images to be identified are identified based on a deep learning network model that has been constructed for imperfect wheat grain images to obtain the grain type of each wheat grain, including:

[0116] Each image to be identified is identified based on a deep learning network model developed for images of imperfect wheat kernels, yielding at least two initial kernel classifications for each kernel. These initial kernel classifications include perfect kernels and imperfect kernels, with the imperfect kernels including insect-damaged kernels, diseased kernels, heat-damaged kernels, broken kernels, sprouted kernels, moldy kernels, and / or shrunken kernels.

[0117] Determine whether at least two initial particles are of the same type.

[0118] If yes, the initial grain type is used as the grain type of the wheat grain. For example, if both images are identified as perfect grains, then it is determined to be a perfect grain.

[0119] If not, the initial grain type with the highest confidence level among the at least two initial grain types that is the imperfect grain is used as the grain type of the wheat grain. For example, if one image identifies a perfect grain and the other image identifies a worm-eaten grain, the imperfect grain type with the highest confidence level (such as a worm-eaten grain) is output. For example, the confidence level of a diseased grain is higher than that of a worm-eaten grain. If one image identifies a diseased grain and the other image identifies a worm-eaten grain, the imperfect grain type with the highest confidence level (such as a diseased grain) is output.

[0120] The embodiment of the present invention determines the category of wheat kernels through a "double authentication" mechanism, thereby improving recognition accuracy.

[0121] In some embodiments of the present invention, the method for identifying imperfect wheat grains further comprises:

[0122] Step S500: Obtain the quantity and weight of wheat in each collection area 16. The quantity of wheat corresponding to each grain type can be counted during identification to determine the quantity of wheat in each collection area 16. A weighing device can be used to determine the weight of the wheat in each collection area 16.

[0123] Step S600 , based on the quantity and weight of the wheat in each collecting area 16 and the quantity and weight of the wheat in the wheat sample to be tested, obtain the quantity ratio and mass ratio of the wheat in each collecting area 16 .

[0124] In the embodiment of the present invention, the number of perfect kernels and imperfect kernels is finally counted, the imperfect kernels are weighed, and their quantity ratio and quality ratio are calculated, thereby providing data support for wheat grade determination and other inspections.

[0125] The flowchart provided in this embodiment is not intended to indicate that the operations of the method will be performed in any particular order, or that all operations of the method are included in all cases. In addition, the method provided in this embodiment may include additional operations. Additional variations may be made to the above method within the scope of the technical ideas provided by the method of this embodiment.

[0126] It should be understood that in some embodiments, each part can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system.

[0127] In some embodiments of the present invention, the apparatus for identifying defective wheat kernels may further include a memory, a processor, and a computer program stored in the memory and running on the processor. When the computer program is executed by the processor, the steps of the method for identifying defective wheat kernels in any of the above embodiments are implemented.

[0128] The computer program for performing the operation of the present invention can be an assembly instruction, an instruction set architecture (ISA) instruction, a machine instruction, a machine-dependent instruction, a microcode, a firmware instruction, a state setting data, a configuration data of an integrated circuit, or a source code or object code written in any combination of one or more programming languages ​​and procedural programming languages. The computer program can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer. In some embodiments, to perform various aspects of the present invention, an electronic circuit including, for example, a programmable logic circuit, a field programmable gate array (FPGA) or a programmable logic array (PLA) can execute computer-readable program instructions by utilizing the state information of the computer-readable program instructions to personalize the electronic circuit.

[0129] For the purposes of the description of this embodiment, a memory is a tangible device capable of retaining and storing a computer program, which may be any device that can contain, store, communicate, propagate, or transmit a program for use with an instruction execution system, device, or apparatus, or in conjunction with such instruction execution systems, devices, or apparatuses. More specific examples of memories (a non-exhaustive list) include the following: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), a memory stick, a floppy disk, a mechanical encoding device, and any suitable combination thereof. The memory also provides temporary storage space for the operation of instructions during operation. The processor is adapted to execute the stored instructions, and the processor may be a single-core processor, a multi-core processor, a computing cluster, or any number of other configurations.

[0130] The device and method for identifying imperfect wheat grains in the embodiments of the present invention significantly reduce the risks of false detection and missed detection caused by single-view occlusion through collaborative imaging of dual cameras 14; the designed lightweight model is superior to mainstream lightweight models in recognition accuracy and parameter quantity, and can be deployed on embedded devices or mobile terminals to meet the needs of efficient detection in low-resource scenarios; it is suitable for grain storage, processing enterprises and quality inspection agencies, and can be combined with national grain and oil inspection standards (such as GB / T5494-2019) to realize automatic statistics of the number and quality of imperfect grains, providing precise inspection technical support for grain procurement, storage, transportation and processing.

[0131] The device and method for identifying imperfect wheat kernels utilize a fully automated process. For standard test samples, the entire process, from feeding, imaging, identification, to sorting, takes only a few minutes, increasing inspection efficiency 20 times compared to manual labor. Recognition accuracy is significantly improved compared to existing models. Dual-view imaging avoids false detections caused by kernel occlusion, and dual-confirmation rules enable more efficient utilization of identification results. The recognition accuracy reached 98.67% on a self-built dataset. The recognition model (a deep learning network model) is lightweight and can be used on resource-constrained PCs, embedded in mobile terminals, and embedded in edge computing devices. The improved, lightweight model, based on MobileNetV3, has only 1.11M parameters and a computational overhead of 0.07 GFLOPS. Image recognition combines count and quality parameters for inspection, providing comprehensive testing metrics compatible with the latest national grain and oil inspection standards. The quality and quantity percentages of grain and oil can be directly output and displayed.

[0132] At this point, those skilled in the art will recognize that, although a number of exemplary embodiments of the present invention have been shown and described in detail herein, many other variations or modifications consistent with the principles of the present invention may be directly determined or derived from the disclosure of the present invention without departing from the spirit and scope of the present invention. Therefore, the scope of the present invention should be understood and deemed to cover all such other variations or modifications.

Claims

1. A method for identifying imperfect wheat grains, characterized in that: include: causing wheat grains in the wheat sample to be tested to fall one by one; During the falling process of each grain of wheat, at least two cameras are used to photograph each grain of wheat, so as to obtain images of at least two sides of each grain of wheat, thereby obtaining at least two images to be identified; Identify at least two of the images to be identified based on a deep learning network model that has been constructed and is suitable for images of imperfect wheat grains, to obtain the type of each grain of wheat; sorting each grain of wheat into a corresponding collection area based on the grain type; obtaining the quantity and weight of wheat in each of the collection areas; Based on the quantity and weight of the wheat in each of the collecting areas and the quantity and weight of the wheat in the wheat sample to be tested, the quantity ratio and mass ratio of the wheat in each of the collecting areas are obtained.

2. The method for identifying imperfect wheat grains according to claim 1, wherein: A deep learning network model suitable for wheat imperfect grain images has been constructed, including a first deep learning network model and a second deep learning network model, wherein the first deep learning network model identifies more types of grains than the second deep learning network model; The step of identifying at least two images to be identified based on the deep learning network model that has been constructed and is applicable to the image of imperfect wheat grains to obtain the type of each grain of wheat includes: Identify at least two of the to-be-identified images based on the first deep learning network model to obtain the grain type of each grain of wheat, until a first preset number of grain types of wheat are obtained; Determining the second deep learning network model according to a first preset number of wheat grain types; Based on the second deep learning network model, each of the remaining wheat images to be identified is identified to obtain the particle type of each remaining wheat grain.

3. The method for identifying imperfect wheat grains according to claim 2, characterized in that: The construction of the first deep learning network model and the construction of the second deep learning network model both include: Build the MobileNetV3 basic network; Add a coordinate attention module before the neck module in the MobileNetV3 base network; Use a dual-branch channel attention module to replace the SE attention module in the MobileNetV3 base network; Use the multi-scale feature fusion module to replace the feature fusion module in the MobileNetV3 basic network to obtain the first deep learning network model and the second deep learning network model; The first deep learning network model and the second deep learning network model are trained using a picture set of wheat grain types to obtain the trained first deep learning network model and the second deep learning network model.

4. The method for identifying imperfect wheat grains according to claim 3, characterized in that: The method of identifying at least two images to be identified based on the deep learning network model that has been constructed and is applicable to the image of imperfect wheat grains to obtain the type of each grain of wheat includes: identifying each image to be identified based on a deep learning network model that has been constructed and is suitable for images of imperfect wheat grains, thereby obtaining at least two initial grain types for each grain of wheat; the initial grain types include perfect grains and imperfect grains, and the imperfect grains include insect-damaged grains, diseased grains, heat-damaged grains, broken grains, sprouted grains, moldy grains, and / or shriveled grains; determining whether at least two of the initial particles are of the same type; If so, the initial grain type is used as the grain type of the wheat; If not, the initial grain type that is the imperfect grain and has the highest confidence among the at least two initial grain types is used as the grain type of the wheat grain.

5. The method for identifying imperfect wheat grains according to claim 2, wherein: Also includes: The lighting devices on the upper side or both sides of the falling path of the wheat are controlled to emit light, so as to provide lighting when the camera takes a picture of each grain of wheat.

6. A device for identifying imperfect wheat grains, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method for identifying imperfect wheat grains according to any one of claims 1 to 5.

7. The device for identifying imperfect wheat grains according to claim 6, characterized in that: Also includes: a feeding device configured to receive a wheat sample to be tested and to cause the wheat in the wheat sample to be tested to fall down grain by grain; a conveyor belt, horizontally arranged at the lower side of the feeding device, configured to receive the wheat from the feeding device and convey the wheat so that the wheat falls from one end of the conveyor belt; a paddle is provided on the conveyor belt; A lighting device is provided on the upper side or the lower side of one end of the conveyor belt; At least two cameras are disposed on the lower side of one end of the conveyor belt and are evenly distributed along the circumference of the falling path; and A separation device is configured to receive the wheat from the conveyor belt and move the wheat to a corresponding collection area.

8. The device for identifying imperfect wheat grains according to claim 7, characterized in that: The separation device comprises: A collection box having two collection areas with openings facing upward; A partition is horizontally arranged on the upper side of the two collecting areas; and the partition is rotatably arranged so that the partition is inclined toward one collecting area or toward the other collecting area, thereby allowing the wheat on the partition to enter the corresponding collecting area.

Images