A method and apparatus for detecting the appearance quality of fresh corn ears

Abstract

This invention discloses a method and apparatus for detecting the appearance quality of fresh corn ears, belonging to the field of fresh corn appearance detection. The appearance quality detection method includes: introducing SS-neck in YOLOv8, where SS-neck uses max pooling convolutional layers to reduce the dimensionality of the feature map of the original fresh corn ear image; using basic units of ShuffleNetV2 to extract feature information from the dimensionality-reduced feature map; using a parameterless attention mechanism layer to enhance feature extraction in both spatial and channel dimensions; using downsampling units of ShuffleNetV2 to downsample the extracted feature information; using lightweight convolutional layers to map a series of sub-feature maps; and then concatenating the extracted sub-feature maps in the channel dimension. This application not only reduces the number of model parameters but also achieves the fastest detection speed, ensures that the extracted corn ear feature information is not lost, and has high detection accuracy, making it suitable for real-time detection tasks.

Description

Technical Field

[0001] This invention relates to the field of appearance inspection of fresh corn, and in particular to a method and apparatus for inspecting the appearance quality of fresh corn ears. Background Technology

[0002] my country is a major producer and consumer of fresh sweet corn, and the price of corn ears is determined by both taste and appearance. In actual production, appearance inspection is mainly done manually by visual inspection. This is labor-intensive, and coupled with long-term fatigue, missed inspections and inadequate inspections frequently occur, leading to numerous problems in subsequent stages of corn production. Appearance defects can be found all over the surface of the corn, requiring 360° inspection without blind spots. Currently, there is a lack of dedicated inspection equipment on the market.

[0003] Fresh sweet corn ears are harvested at the milk stage, with a moisture content of over 60%, thin skin, and abundant juice, making them extremely susceptible to damage during production. Currently, there are related visual inspection technologies for mature ears and variety evaluation, but the material properties differ significantly from those of fresh sweet corn ears. For example, patent application number 2012105716413, entitled "Method for Stitching Ordered Images of Corn Ears," primarily focuses on stitching ordered images of mature ears for variety evaluation calculations; patent application number 2015104017970, entitled "Method and System for Variety Evaluation of Corn Ears," designs a method of rotating ears using double rollers, but the ears are in a rotating state throughout the entire operating cycle, especially at high speeds, making it unsuitable for fresh sweet corn; patent application number 2010102881528, entitled "Computer Vision Inspection and Grading Method and Device for Fresh Corn Ear Quality," discloses a method for grading fresh sweet corn ears using machine vision to acquire images, but the ears are in a static state, failing to meet the practical requirement of a full rotation.

[0004] To achieve complete surface inspection of corn ears, a combination of inspection technology and equipment processes is needed. This requires ensuring the acquisition of all surface information while minimizing hard contact. Existing visual inspection technologies lack sufficient protection for the material, resulting in the material being constantly in a state of friction or hard contact, making them unsuitable for long-term appearance inspection during the processing of fresh corn ears.

[0005] In addition, traditional machine vision technology is widely used for the detection of corn ears and seeds, which is mainly achieved by combining image processing technology, human feature extraction and classification algorithms. Traditional machine learning technology achieves classification by extracting effective features such as texture, color and size of corn ears and seeds. Although it can achieve automatic recognition and detection, the manual feature extraction required is laborious and time-consuming. Furthermore, the robustness of the model deteriorates as the ear variety changes, the detection accuracy decreases, and the detection speed is slow.

[0006] Existing technologies utilize the NASNet-mobile network and apply transfer learning methods to classify and identify maize varieties. However, this method requires adjustments to model parameters and structure as varieties change, presenting certain limitations and making it unsuitable for deployment on mobile devices. While improved GAN networks can be used for data augmentation, and a maize ear detection model can be proposed by combining transfer learning, although its detection accuracy is higher than other models, it suffers from a large number of parameters, making training difficult, resulting in slow detection speeds and making it unsuitable for deployment on edge devices. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this invention provides a method and apparatus for detecting the appearance quality of fresh corn ears. It is suitable for real-time, non-destructive testing of ears during high-speed movement in factory processing lines. It enables 360° appearance quality testing of ears during processing and transportation, replacing manual inspection, improving production efficiency and product marketability. Furthermore, it is adaptable to the simultaneous detection of large ear sizes and small surface defects in fresh corn ears. The lightweight design of the YOLOv8 backbone network ensures the fastest detection speed, making it suitable for real-time detection tasks.

[0008] The present invention adopts the following technical solution:

[0009] The first aspect of this invention discloses a method for detecting the appearance quality of fresh corn ears, comprising the following steps:

[0010] A set of fresh corn cob images was acquired using a camera. The images were then processed: SS-neck was introduced in YOLOv8. SS-neck uses max-pooling convolutional layers to reduce the dimensionality of the original fresh corn cob image feature maps, resulting in dimensionality-reduced feature maps. ShuffleNetV2 basic units were used to extract feature information from the dimensionality-reduced feature maps. A parameterless attention mechanism layer was used to enhance feature extraction in both spatial and channel dimensions. ShuffleNetV2 downsampling units were used to downsample the extracted feature information. The feature maps were then repeatedly processed through the parameterless attention mechanism layer, ShuffleNetV2 downsampling units, and ShuffleNetV2 basic units to obtain channel-washed feature maps. Lightweight convolutional layers were used to map a series of sub-feature maps. These sub-feature maps were then concatenated along the channel dimension to obtain a feature map with increased channel dimension and reduced spatial dimension. An SPPF module was used to obtain the final feature map. The neck network fused the feature information extracted by SS-neck, ensuring sufficient feature integration. The head detection network then used the fused feature information from the neck network for regression decision-making, ultimately outputting the detected defect results.

[0011] According to the method for detecting the appearance quality of fresh corn ears, the camera starts a multi-threaded tracking and shooting mode, while the cable chain drives the tray to rotate clockwise. The camera contains m threads, where m is not less than four. Thread 1 is used to acquire images, and the other threads are used to process images. When the tray moves to the left edge of the camera's field of view, Thread 1 acquires image Pn1 at time Tn1. The camera takes pictures at time intervals of T. After capturing an image, Thread 1 passes it to Thread 2, which processes the image acquired by Thread 1 during the waiting time. At the same time, Thread 1 keeps timing itself and continues to capture the next image at time Tn2. After n-1 intervals T, Thread 1 acquires n images Pn1-Pnn, where each image Pnn contains n trays corresponding to ears of corn 1-n, which are passed to Thread 2-m respectively. When ear 1 moves to the right edge of the camera's field of view, the acquisition of image ear 1 ends. After the statistics are completed, Thread 2 is released for the processing of image ear n+1 in the next round.

[0012] According to the method for detecting the appearance quality of fresh corn ears, the ears are placed in a tray and move with the tray. At this time, the ears are relatively stationary relative to the tray. The tray continues to move, and the ears come into contact with the inclined surface of the guide rail. After passing the inclined surface of the guide rail, the ears float on the guide rail. The inner wall on the left side of the tray pushes the ears, and the ears rotate clockwise under the action of the guide rail.

[0013] According to the method for detecting the appearance quality of fresh corn ears, thread one acquires image Pn2 at time Tn2. At this time, the image contains ear one and ear two. Ear one is a second-angle image, and ear two is a first-angle image. Ear one is then passed to thread two, and ear two is passed to thread three. At time Tn3, thread one acquires image Pn3. At this time, the image contains ear one, ear two, and ear three. Ear one is a third-angle image, ear two is a second-angle image, and ear three is a first-angle image. Ear one is then passed to thread two, ear two is passed to thread three, and ear three is passed to thread four.

[0014] According to the method for detecting the appearance quality of fresh corn ears, the ears are photographed n times, with each photograph taken at an angle of [angle value missing]. Thread 2 processes each of the n images and counts the defects in each image.

[0015] According to the aforementioned method for detecting the appearance quality of fresh corn ears, the max pooling convolutional layer includes: a convolutional structure, a batch normalization structure, a ReLU activation function, and a max pooling structure. The max pooling convolutional layer first extracts features through a 3×3 convolutional structure, then normalizes them through a batch normalization structure, then activates them through a ReLU activation function, and finally obtains the dimensionality-reduced feature map through a 3×3 max pooling operation.

[0016] According to the aforementioned method for detecting the appearance quality of fresh corn ears, the information of several feature channels in the dimensionality-reduced feature map is subjected to channel separation. After separation, the information is entered into the identity mapping path and the convolution re-extraction path respectively. The feature information of the convolution re-extraction path is re-extracted by convolution, and the feature information of the identity mapping path is directly subjected to identity mapping. The two feature information of identity mapping and convolution re-extraction are concatenated by the Concat function, and then channel shuffling is performed to obtain the feature map after feature fusion.

[0017] According to the aforementioned method for detecting the appearance quality of fresh corn ears, the convolutional re-extraction path first performs dimensionality reduction on the channel feature information through a 1×1 convolution, then through a 3×3 depthwise convolution, and finally through a 1×1 convolution to increase the dimensionality, while keeping the number of output feature channels unchanged.

[0018] According to the method for detecting the appearance quality of fresh corn ears, the enhanced channel feature information flows into the first convolution branch and the second convolution branch respectively, performs feature information downsampling, then concatenates them through the Concat function, and then performs channel shuffling. After downsampling, the number of output channels of the feature map is doubled and the size is halved. The first convolution branch includes: 3×3 depth convolution and 1×1 ordinary convolution in sequence; the second convolution branch includes: 1×1 ordinary convolution, 3×3 depth convolution and 1×1 ordinary convolution in sequence.

[0019] According to the aforementioned method for detecting the appearance quality of fresh corn ears, the characteristics after channel washing are obtained. Figure X ,feature Figure X The size is S×S×C1, and a series of sub-features are mapped through lightweight convolutional layers. Figure X ', Sub-feature Figure X The size of the feature map is S'×S'×C1. When the scaling factor is 2, there will be 4 sub-feature maps. Connecting the 4 mapped feature maps along the channel dimension will result in a feature map with the channel dimension increased by 4 times and the spatial dimension reduced by 2 times. Figure X Then, the feature map is input into a non-stepping convolutional layer with C2 filters, and finally, the output feature map size is S / 2×S / 2×C2.

[0020] Another aspect of this application provides a device for inspecting the appearance quality of fresh corn ears, including: a machine support, a cable chain, a tray, a guide rail mechanism, and a camera; a cable chain is provided on the top of the machine support, and trays are fixed at equal intervals above the cable chain; a camera bracket is provided at the middle position of the top of the machine support, and the camera is fixed on the crossbar of the camera bracket; the vertical direction of the camera is perpendicular to the cable chain, and the camera is located on the central plane of the guide rail mechanism.

[0021] According to the aforementioned fresh corn ear appearance quality inspection device, the cable chain includes: a first cable chain, a second cable chain, a front gear, a rear gear, a front connecting shaft, and a rear connecting shaft; the front gear includes: a first cable chain front gear and a second cable chain front gear, the first cable chain front gear and the second cable chain front gear being connected via the front connecting shaft; the rear gear includes: a first cable chain rear gear and a second cable chain rear gear, the first cable chain rear gear and the second cable chain rear gear being connected via the rear connecting shaft; the extended portions of the front connecting shaft and the rear connecting shaft are connected to the overall machine support; the protruding portion of the front connecting shaft can be externally connected to a power unit.

[0022] According to the aforementioned fresh corn cob appearance quality inspection device, the guide rail mechanism includes: a guide rail, a guide rail bracket, and a support; the guide rail is a long rod shape, with inclined surfaces at both ends, and the upper and lower sides of the guide rail are flat, with equally spaced grooves on the upper surface, and at least two mounting holes equally spaced along the axial direction on the upper surface, and the lower surface is connected to the guide rail bracket by a connector; the guide rail bracket sits on the support, and the support is connected to the overall machine bracket.

[0023] According to the aforementioned fresh corn ear appearance quality inspection device, the tray is a long semi-cylindrical shape with an operating groove in the middle for cooperating with the guide rail to make the ears roll. The two sides are arc-shaped for limiting the position of the ears. The bottom is provided with a boss for connecting with the cable chain. The groove surface of the operating groove is higher than the bottom of the tray.

[0024] According to the aforementioned fresh corn cob appearance quality inspection device, the guide rail mechanism is provided with at least one guide rail, which is located below the tray. The upper planes of all guide rails are at the same height, the two sides of the guide rail are parallel to the plane where the cable chain is located, and the two sides of the guide rail are perpendicular to the tray. The upper plane of the guide rail is higher than the bottom of the tray and lower than the operating groove surface in the middle of the tray.

[0025] Compared with the prior art, the beneficial effects of the present invention include at least the following:

[0026] (1) This application targets high-speed moving ears of fruit during processing. Utilizing the principle of space division multiplexing and multi-threaded processing strategy, this application designs the structure of the cable chain, pallet, and guide rail mechanism. The cable chain, pallet, and guide rail mechanism work together to achieve 360° rotation of the ears of fruit during processing and conveying, and the pallet space is reasonably allocated. The camera can process images of the ears of fruit from various angles in a short time through the multi-threaded processing strategy, which increases the timeliness of visual inspection.

[0027] (2) This application adopts a lightweight design for the backbone network, and introduces ShuffleNetV2 downsampling units, ShuffleNetV2 basic units, max pooling convolution and lightweight convolutional layers. This not only reduces the number of model parameters, but also has the fastest detection speed, ensuring that the extracted corn ear feature information is not lost, and has high detection accuracy, which is suitable for real-time detection tasks.

[0028] (3) This application introduces the SimAM module, which unifies the feature weights by directly calculating the three-dimensional attention weights in the feature map, thereby enhancing the model's feature focusing ability on the features of defective ears;

[0029] (4) This application introduces Wise-IoU, which improves the convergence speed and reduces the impact of low-quality samples on the model's generalization ability;

[0030] (5) This application designs a long strip tray for fresh corn cob material, with a hollow center and curved sides, and with the guide rail designed at the bottom, so that the material can float up and rotate evenly during the visual detection area.

[0031] (6) Within the effective area of ​​visual inspection, the frame rate of the camera is matched with the speed of the pallet movement to achieve material tracking, acquisition of multiple images at a fixed angle, and quality inspection. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of the testing method in this application;

[0033] Figure 2 This is a diagram of the YOLOv8 network architecture in this application;

[0034] Figure 3 This is a diagram of the maximum pooling layer structure in this application;

[0035] Figure 4 This is a diagram of the lightweight convolutional layer structure in this application;

[0036] Figure 5 This is a comparison chart of the convergence of the loss curves;

[0037] Figure 6 This is a three-dimensional structural diagram of the testing device of this application;

[0038] Figure 7 This is a schematic diagram of the tray used in this application;

[0039] Figure 8 This is a schematic diagram of the guide rail in this application;

[0040] Figure 9 This is a side view of the single pallet and guide rail mechanism assembled according to this application;

[0041] Figure 10This is a top view of a single tray assembled with guide rails according to this application;

[0042] Figure 11 This is a diagram illustrating the interaction between the ear of fruit, the tray, and the guide rail in this application.

[0043] In the diagram: 1. Main unit bracket; 2. Cable carrier; 21. First cable carrier; 22. Second cable carrier; 23. Front gear; 24. Rear gear; 25. Front connecting shaft; 26. Rear connecting shaft; 3. Tray; 31. Operating slot; 32. Countersunk positioning hole; 33. Boss; 4. Guide rail mechanism; 41. Guide rail; 42. Guide rail bracket; 43. Support; 44. Mounting hole; 5. Camera; 51. Camera bracket. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of this invention. The embodiments described in this application are merely some embodiments of this invention, and not all embodiments. Based on the spirit of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of this invention.

[0045] In the description of this invention, it should be noted that the terms "front," "rear," "inner," "outer," "right," "left," "both ends," "one end," and "the other end," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention. In addition, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. The term "clockwise" is limited to descriptive purposes and should not be construed as meaning that this device can only rotate clockwise.

[0046] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installed," "equipped with," "connected," etc., should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be a connection within two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0047] Example 1:

[0048] like Figures 1-2 As shown, the present invention provides a method for detecting the appearance quality of fresh corn ears, comprising the following steps:

[0049] S1. Acquire a set of images of fresh corn ears. The image set is mainly acquired through camera 5. In other embodiments, the image set can also be acquired through a shooting device such as a mobile phone. Camera 5 starts a multi-threaded tracking shooting mode. Camera 5 contains m threads, where the value of m is not less than four. Thread one is responsible for acquiring images, and the other threads are responsible for processing images. At the same time, the cable chain 2 drives the tray 3 to rotate clockwise at a speed of v.

[0050] S1.1 Fresh corn ears are placed in tray 3 and move together with tray 3. At this time, the ears are relatively stationary relative to tray 3. The radius of the ears is r, and the circumference is S = 2πr.

[0051] S1.2, tray 3 continues to move, the ear of fruit contacts the inclined surface of guide rail 41, and after passing the inclined surface of guide rail 41, it floats on guide rail 41. The length of guide rail 41 is L.

[0052] S1.3 The inner wall on the left side of the tray 3 pushes the fruit bunch, and the fruit bunch rotates clockwise under the action of the guide rail 41.

[0053] S1.4, tray 3 moves to the left edge of camera 5's field of view. Camera 5 thread 1 acquires image Pn1 at time Tn1, which only contains the first angle image of ear 1, and the image covers the entire surface of the ear. n = m - 1.

[0054] S1.5 Camera 5 takes pictures at timed intervals of T. After capturing an image, thread 1 passes it to thread 2. Thread 2 processes the image acquired by thread 1 during the waiting time. At the same time, thread 1 starts its timer and continues to capture the next image at time Tn2.

[0055] S1.6 Thread 1 obtains the image Pn2 at time Tn2. At this time, the image contains ear of fruit 1 and ear of fruit 2. Ear of fruit 1 is the second angle image and ear of fruit 2 is the first angle image. Ear of fruit 1 is passed to thread 2, and ear of fruit 2 is passed to thread 3.

[0056] S1.7. Thread 1 acquires image Pn3 at time Tn3. At this moment, the image contains ears of fruit 1, ear of fruit 2, and ear of fruit 3. Ear of fruit 1 is a third-angle image, ear of fruit 2 is a second-angle image, and ear of fruit 3 is a first-angle image. Ear of fruit 1 is then passed to thread 2, ear of fruit 2 is passed to thread 3, and ear of fruit 3 is passed to thread 4. And so on. After n-1 intervals T, thread 1 acquires n images Pn1-Pnn. Image Pnn contains n trays 3 corresponding to ear of fruit 1-ear n, which are then passed to threads 2-m respectively.

[0057] S1.8, when ear of fruit one moves to the right edge of the field of view of camera 5, the image acquisition of ear of fruit one ends. Ear of fruit one is photographed a total of n times, with each photograph taken at an angle of... Thread 2 processes each of the n images, calculating the defect status of each image. After the calculation is complete, thread 2 is released and used for the next round of processing the n+1th image cluster.

[0058] like Figure 1 As shown, in this embodiment, the value of m is seven, and the value of n is six, meaning that camera 5 contains seven threads. Thread 1 is responsible for acquiring images, and threads 2 through 7 are responsible for processing the images. Thread 1 acquires image Pn1 at time Tn1, which only contains the first angle image of ear of fruit 1, and the image occupies 1 / 6 of the entire ear of fruit surface. Thread 1 acquires six images Pn1-Pn6 after six time intervals T. Image Pn6 contains six trays corresponding to ear of fruit 1 through ear of fruit 6, and these images are passed to threads 2 through 7 respectively. Ear of fruit 1 is photographed a total of six times, each time at an angle of 60°. Thread 2 processes each of the six images and calculates the defect status of each image. After the calculation is completed, thread 2 is released and used for the next round of processing of image ear of fruit 7.

[0059] In another embodiment, camera 5 includes thirteen threads: thread one is responsible for acquiring images, and threads two through thirteen are responsible for processing images. The ear of fruit is photographed a total of twelve times, each time at an angle of 30°.

[0060] In another embodiment, camera 5 includes four threads: thread one is responsible for acquiring images, and threads two through four are responsible for processing images. The ear of fruit is photographed a total of twelve times, each time at an angle of 120°.

[0061] Data constraints:

[0062] (1) L>S, the optimal ratio is L=1.5S=3πr;

[0063] (2) vT = S / N = 2πr / N.

[0064] Example 2:

[0065] This application improves the backbone feature extraction network based on the characteristic that fresh corn ears are composed of a large number of independently and tightly arranged kernels. It adopts a strategy that combines ShuffleNetV2 for feature reuse, lightweight convolutional layers that ensure fine-grained information, and max-pooling convolutional layers that reduce computational cost. This ensures that the model does not lose local and fine-grained information about the ear, while also achieving model lightweighting. Addressing the issue that the proportion of missing and detached kernels in inferior fresh corn ears is small, leading to false detections due to the backbone feature extraction network's failure to capture these features, a parameter-free attention module is introduced into the backbone feature extraction network module to improve the model's ability to extract features from ears with missing or detached kernels. Furthermore, Wise-IoU is introduced as the bounding box regression loss function to compensate for the shortcomings of the Complete-IoU loss function, where the predicted box length and width cannot change simultaneously, resulting in large size differences in deformed ears that affect training convergence speed and model performance, thus further ensuring the detection performance of this application.

[0066] Images of superior and inferior fresh corn ears were collected from the production line using the method described in Example 1. A dataset of superior and inferior fresh corn ears was established as the data for this study. Superior ears include first-grade and second-grade ears, while inferior ears include deformed, missing, missing, and mechanically damaged ears.

[0067] Figure 2 As shown, the obtained image is processed as follows: SS-neck is introduced in YOLOv8. SS-neck uses max pooling convolutional layers to reduce the dimensionality of the feature map of the original fresh corn cob image, resulting in a dimensionality-reduced feature map. ShuffleNetV2 basic units are used to extract feature information from the dimensionality-reduced feature map. A parameterless attention mechanism layer is used to enhance the extraction of spatial and channel dimensions. ShuffleNetV2 downsampling units are used to downsample the extracted feature information. The feature map is then repeatedly passed through the parameterless attention mechanism layer, the ShuffleNetV2 downsampling unit, and the ShuffleNetV2 basic units to obtain a channel-washed feature map. A lightweight convolutional layer is used to map a series of sub-feature maps. The extracted sub-feature maps are then concatenated along the channel dimension to obtain a feature map with increased channel dimension and reduced spatial dimension. The SPPF module is used to obtain the final feature map.

[0068] S2.1, such as Figure 3As shown, the input layer receives the image acquired by camera 5, resizes the image to a predetermined size, such as 640×640, and inputs the resized image into a max-pooling convolutional layer. The max-pooling convolutional layer includes a convolutional structure, a batch normalization structure, a ReLU activation function, and a max-pooling structure. The max-pooling convolutional layer first extracts features through a 3×3 convolutional structure, then normalizes them through a batch normalization structure, then activates them through a ReLU activation function, and finally performs a 3×3 max-pooling operation to obtain a dimensionality-reduced feature map with C channels.

[0069] In this process, the convolutional structure first extracts the features of the input corn ear, and the output feature map enters the max pooling structure for downsampling to further extract feature information. Compared with the convolutional layer, the max pooling structure can not only complete the downsampling operation, but also reduce the amount of computation.

[0070] S2.2 Input the dimensionality-reduced feature map into the ShuffleNetV2 basic unit to obtain the feature map after feature fusion;

[0071] S2.2.1 Perform channel separation on the information of C feature channels, and then enter the identity mapping path and the convolution re-extraction path respectively; let the number of feature channels in the identity mapping path be C3, and the number of feature channels in the convolution re-extraction path be C4.

[0072] Where C3 + C4 = C;

[0073] The S2.2.2.1 and C2 channel feature information are first reduced in dimensionality by 1×1 convolution, then by 3×3 depthwise convolution, and finally by 1×1 convolution to increase the dimensionality, while the number of output feature channels remains unchanged.

[0074] The characteristic information of channels S2.2.2.2 and C1 is directly mapped using an identity method;

[0075] S2.2.3. The two feature information extracted by identity mapping and convolution are concatenated by the Concat function to ensure that the extracted feature information is not lost. Then, channel shuffling is performed to ensure that the obtained feature information is fully fused to obtain the feature map after feature fusion.

[0076] S2.3. The feature map after feature fusion is input into the parameterless attention mechanism layer to enhance feature extraction in both spatial and channel dimensions, thereby enhancing the ability to represent feature information and directly calculating the three-dimensional attention weights. The parameterless attention mechanism layer directly calculates the feature weights of the feature map, assigning a unique weight to each neuron. Each neuron has an independent energy function. Minimizing this energy function ultimately yields the minimum energy function for a single neuron.

[0077]

[0078] In the formula and It is the mean and variance of all neurons in the channel except t; as can be seen from the above formula, the lower the energy, the more obvious the distinction between neuron t and the surrounding neurons, the more significant the feature attention, and the enhanced feature information representation ability.

[0079] It can improve the model's attention to features such as missing kernels, mechanical damage, and kernel loss in inferior ears of grain;

[0080] S2.4. The enhanced feature information is input into the ShuffleNetV2 downsampling unit. The channel feature information flows into the first convolution branch and the second convolution branch respectively to downsample the feature information. Then, it is concatenated by the Concat function and then channel shuffling is performed. After downsampling, the number of output channels of the feature map is doubled and the size is halved to ensure that the extracted feature information is not lost.

[0081] The first convolutional branch includes, in sequence, a 3×3 depthwise convolution and a 1×1 ordinary convolution; the second convolutional branch includes, in sequence, a 1×1 ordinary convolution, a 3×3 depthwise convolution and a 1×1 ordinary convolution.

[0082] S2.5, such as Figure 4 As shown, steps S2.3, S2.4, and S2.2 are repeated sequentially to obtain the characteristics after channel shuffling. Figure X Assuming its size is S×S×C1, a series of sub-features will be mapped out through a lightweight convolutional layer. Figure X The size is S'×S'×C1. When the scaling factor is 2, there will be 4 sub-feature maps. Connecting the 4 mapped feature maps along the channel dimension results in a feature map with the channel dimension increased by 4 times and the spatial dimension decreased by 2 times. Figure X The size of “X” is S / 2×S / 2×4C1. Then, this feature map is input into a non-staggered convolutional layer with C2 filters. Finally, the size of the output feature map is S / 2×S / 2×C2.

[0083] The lightweight convolutional layer includes a spatial-to-depth layer and a non-staggered convolutional layer. The spatial-to-depth layer downsamples the feature information of fresh corn ears extracted by ShuffleNetV2 while retaining all feature information in the channel dimension, thereby avoiding feature loss. The non-staggered convolutional layer can achieve feature extraction without reducing the size of the feature map, which can further reduce the loss of fine-grained information in the corn ear image, thereby ensuring the accuracy of the network model.

[0084] S2.6. Input the feature information of the feature map obtained above into the SPPF module, perform 1×1 convolution operation to halve the channels in sequence, then perform pooling operation with kernel size of 5, 9 and 13 in sequence, then concatenate the pooled feature information, and finally perform 1×1 convolution operation to adjust the channels to obtain the concatenated feature map.

[0085] S2.7 The neck network fuses the feature information extracted from the backbone features, so that the feature information is fully integrated;

[0086] S2.8 The head network detects the feature information fused from the neck network and performs regression decision-making to finally output the detected defect results.

[0087] The SS-neck structure in this application uses the ShuffleNetV2 structure for feature extraction, avoiding the loss of feature information when the backbone network is halved. The convolutional structure using max pooling convolutional layers and lightweight convolutional layers for downsampling not only reduces the number of model parameters, but also ensures that the extracted corn ear feature information is not lost, thus guaranteeing high-precision detection by the model.

[0088] like Figure 2 As shown, the dashed box represents the backbone network of this application. The original model backbone network is replaced by SS-neck. Input represents the input layer, Conv_Maxpool represents the max pooling convolutional layer, ShuffleNetV2_B represents the ShuffleNetV2 basic unit, SimAM represents the parameterless attention mechanism layer, and ShuffleNetV2_U represents the ShuffleNetV2 downsampling unit. SPDConv represents the lightweight convolutional layer, SPPF represents the SPPF module in the YOLOv8 network, concat represents the concat function, Upsample represents the Upsample function, Detect represents the Detect layer in the YOLOv8 network, conv represents the convolutional layer, and C2f represents the C2f module.

[0089] The activation functions for max-pooling convolutional layers, ShuffleNetV2 basic units, and ShuffleNetV2 downsampling units are ReLU functions.

[0090] In the low-quality samples of fresh corn ears, the malformed ears have a large size difference compared to other ears. If the length and width of the prediction box cannot be changed simultaneously, it will affect the training results. This application uses the Wise-IoU loss function, which has a dynamic gradient gain allocation strategy and separate length and width calculation. Wise-IoU can dynamically adjust the loss function according to the quality of the sample through a dynamic non-monotonic focusing gradient gain allocation strategy, reduce the competitive advantage of high-quality samples, mitigate the adverse gradient effects caused by low-quality samples, and improve the detection accuracy and robustness of the model. The Wise-IoU with two attention layers is obtained according to the distance metric, as shown in Equation (2):

[0091]

[0092]

[0093] Where R WIoU For W IoU The distance metric function is defined as shown in equation (4); the L of the ordinary mass anchor frame will be significantly magnified. IoU (Wg, Hg) represent the length and width of the optimal bounding box, respectively; * indicates the separation operation to prevent R from being delimited. WIoU This generates a gradient that hinders convergence, separates Wg and Hg from the calculation, avoids the situation where the length and width of the prediction box cannot change simultaneously, and reduces interference with model training.

[0094] like Figure 5 As shown, the red and black curves represent WIoU and CIoU respectively, while the blue, green, and purple curves represent DIoU, GIoU, and SIoU respectively. It can be seen that after introducing Wise-IoU, the model converges around 50 epochs with good convergence, while the other four loss functions converge after nearly 180 epochs, and the loss value during training is much lower than the other four loss functions.

[0095] Table 1. Comparison of detection performance of YOLO detection models

[0096]

[0097] Table 1 shows that the improved model achieves higher precision, recall, and mAP compared to YOLOv5 and YOLOv8. Compared to YOLOv6, the precision and recall are slightly lower, with reductions within 1 percentage point. Compared to other detection models, the improved model has a smaller size and fewer computational parameters, with parameter counts reduced by 0.52, 1.02, and 2.25 MB, respectively; and model size reduced by 1.3, 2.3, and 4.7 MB, respectively. Considering both detection accuracy, computational complexity, and size, the improved model is more suitable for maize ear quality detection.

[0098] Example 3:

[0099] like Figure 6-11 As shown, the present invention also provides a device for inspecting the appearance quality of fresh corn ears, including: a machine support 1, a cable chain 2, a tray 3, a guide rail mechanism 4, and a camera 5. A cable chain 2 is mounted on the top of the machine support 1, and a tray 3 is fixed above the cable chain 2. A camera bracket 51 is mounted at the middle of the top of the machine support 1, and the camera 5 is fixed to the horizontal bar of the camera bracket 51. The vertical direction of the camera 5 is perpendicular to the cable chain 2, and the camera 5 is located on the central plane of the guide rail mechanism 4.

[0100] The frame provides support for the device and is made of stainless steel.

[0101] like Figure 6 As shown, the cable chain 2 includes: a first cable chain 21, a second cable chain 22, a front gear 23, a rear gear 24, a front connecting shaft 25, and a rear connecting shaft 26; the front gear 23 includes: a first cable chain front gear and a second cable chain front gear, which are connected by the front connecting shaft 25; the rear gear 24 includes: a first cable chain rear gear and a second cable chain rear gear, which are connected by the rear connecting shaft 26. The extended portions of the front connecting shaft 25 and the rear connecting shaft 26 are connected to the main frame 1; trays 3 are installed at equal intervals on the top of the cable chain 2, and the cable chain 2 sits on the main frame 1 to support the trays 3. The protruding portion of the front connecting shaft 25 protrudes outward by 10cm for external connection to a power unit.

[0102] like Figure 7 As shown, the tray 3 is a long, semi-cylindrical shape with an operating groove 31 in the middle for cooperating with the guide rail 41 to tumble the fruit bunches. The two sides are arc-shaped for limiting the position of the fruit bunches. The front and rear ends of the tray 3 have countersunk positioning holes 32, and the bottom is provided with a boss 33 for connecting with the cable chain 2. The groove surface of the operating groove 31 is higher than the bottom of the tray 3.

[0103] like Figure 8 or Figure 9As shown, the guide rail mechanism 4 includes: a guide rail 41, a guide rail bracket 42, and a support 43. The guide rail 41 can be made of silicone material and is a long rod shape. The guide rail 41 has inclined surfaces at both ends, and its upper and lower sides are flat. The upper surface has equally spaced grooves, and at least two mounting holes 44 are equally spaced along the axial direction. The lower surface is connected to the guide rail bracket 42 by bolts or pins. In this embodiment, three mounting holes 44 are provided to fix the guide rail 41 to the guide rail bracket 42. In another embodiment, three mounting holes 44 can be provided for fixing the guide rail 41 to the guide rail bracket 42. The equally spaced grooves increase friction and ensure uniform rotation of the ear of fruit. The guide rail bracket 42 sits on the support 43, which is connected to the machine frame 1.

[0104] The tray 3 works in conjunction with the guide rail mechanism 4 to achieve the rotation of the fruit bunch.

[0105] The guide rail mechanism 4 is provided with at least one guide rail 41. In this embodiment, two guide rails 41 are provided, one at each end and placed in parallel. The guide rails 41 are located below the tray 3, and the upper surfaces of the guide rails 41 are at the same height. The two sides of the guide rails 41 are parallel to the plane of the cable chain 2 and perpendicular to the tray 3. The distance between the two guide rails 41 can be adjusted according to the length of the fruit bunch, generally around 20cm. In another embodiment, a single guide rail 41 can be provided, which is placed at the center below the tray 3 and perpendicular to the tray 3.

[0106] like Figure 11 As shown, the upper surface of the guide rail 41 is higher than the bottom of the tray 3 and lower than the operating groove 31 in the middle of the tray 3. The upper surface is higher than the bottom of the tray 3 to support the fruit bunch, and the lower surface is lower than the operating groove 31 in the middle of the tray 3 to allow the surface to scrape against the tray 3, so that the fruit bunch can rotate as the tray 3 moves on the guide rail 41.

[0107] This invention discloses a method and apparatus for detecting the appearance quality of fresh corn ears. The method includes using a cable chain 2 to drive a tray 3 within a limited length, and using the interaction between the tray 3, guide rail 41, and the ears to synchronously achieve forward movement and uniform rotation. A camera 5 acquires a certain number of fresh corn images within a suitable field of view. Using the principle of spatial division multiplexing and a multi-threaded processing strategy, images of each ear from different angles are acquired and processed within the shooting cycle to obtain quality data, thereby achieving rapid detection of the appearance quality of fresh corn.

[0108] Compared with the prior art, the beneficial effects of the present invention include at least the following:

[0109] (1) This application targets high-speed moving ears of fruit during processing. Utilizing the principle of space division multiplexing and multi-threaded processing strategy, this application designs the structure of the cable chain, pallet, and guide rail mechanism. The cable chain, pallet, and guide rail mechanism work together to achieve 360° rotation of the ears of fruit during processing and conveying, and the pallet space is reasonably allocated. The camera can process images of the ears of fruit from various angles in a short time through the multi-threaded processing strategy, which increases the timeliness of visual inspection.

[0110] (2) This application adopts a lightweight design for the backbone network, and introduces ShuffleNetV2 downsampling units, ShuffleNetV2 basic units, max pooling convolution and lightweight convolutional layers. This not only reduces the number of model parameters, but also has the fastest detection speed, ensuring that the extracted corn ear feature information is not lost, and has high detection accuracy, which is suitable for real-time detection tasks.

[0111] (3) This application introduces the SimAM module, which unifies the feature weights by directly calculating the three-dimensional attention weights in the feature map, thereby enhancing the model's feature focusing ability on the features of defective ears;

[0112] (4) This application introduces Wise-IoU, which improves the convergence speed and reduces the impact of low-quality samples on the model's generalization ability;

[0113] (5) This application designs a long strip tray for fresh corn cob material, with a hollow center and curved sides, and with the guide rail designed at the bottom, so that the material can float up and rotate evenly during the visual detection area.

[0114] (6) Within the effective area of ​​visual inspection, the frame rate of the camera is matched with the speed of the pallet movement to achieve material tracking, acquisition of multiple images at a fixed angle, and quality inspection.

[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the claims of the present invention.

Claims

1. A method for detecting the appearance quality of fresh corn ears, characterized in that, Includes the following steps: The camera (5) acquires a set of fresh corn cob images; the acquired images are processed as follows: SS-neck is introduced in YOLOv8. SS-neck uses max pooling convolutional layers to reduce the dimension of the feature map of the original fresh corn cob image, and obtains the dimension-reduced feature map. The basic unit of ShuffleNetV2 is used to extract feature information from the dimension-reduced feature map. The parameterless attention mechanism layer is used to enhance the feature extraction of spatial dimension and channel dimension. The downsampling unit of ShuffleNetV2 is used to downsample the extracted feature information. The feature map is repeatedly passed through the parameterless attention mechanism layer, the downsampling unit of ShuffleNetV2 and the basic unit of ShuffleNetV2 to obtain the feature map after channel shuffling. The lightweight convolutional layer is used to map out a series of sub-feature maps. The extracted sub-feature maps are then spliced ​​in the channel dimension to obtain a feature map with increased channel dimension and reduced spatial dimension. The SPPF module is used to obtain the final feature map. The neck network fuses the feature information extracted by SS-neck features to make the feature information fully fused. The head network detects the feature information fused with the neck network for regression decision-making, and finally outputs the detected defect results; the camera (5) starts the multi-threaded tracking shooting mode, and at the same time the drag chain (2) drives the tray (3) to rotate clockwise. The camera (5) contains m threads, where the value of m is not less than four. Thread one is used to acquire images, and the other threads are used to process images; the tray (3) moves to the left edge of the field of view of the camera (5), and the first thread of the camera (5) acquires the Pn1 image at time Tn1; the camera (5) takes pictures at time intervals of T; the first thread captures... After obtaining the image, it is passed to thread 2. Thread 2 processes the image obtained by thread 1 during the waiting time. At the same time, thread 1 is timed and continues to capture the next image at time Tn2. After n-1 intervals T, thread 1 obtains n images Pn1-Pnn. There are n trays (3) in the Pnn image, which correspond to ear of fruit 1-ear n respectively. They are passed to thread 2-thread m respectively. When ear of fruit 1 moves to the right boundary of the camera's (5) field of view, the acquisition of the image of ear of fruit 1 ends. After the statistics are completed, thread 2 is released and used for the processing of the next round of image ear of fruit 1. Thread 1 acquires image Pn2 at time Tn2. At this time, the image contains ear of fruit 1 and ear of fruit 2. Ear of fruit 1 is a second-angle image, and ear of fruit 2 is a first-angle image. Ear of fruit 1 is then passed to thread 2, and ear of fruit 2 is then passed to thread 3. Thread 1 acquires image Pn3 at time Tn3. At this time, the image contains ear of fruit 1, ear of fruit 2, and ear of fruit 3. Ear of fruit 1 is a third-angle image, ear of fruit 2 is a second-angle image, and ear of fruit 3 is a first-angle image. Ear of fruit 1 is then passed to thread 2, ear of fruit 2 is then passed to thread 3, and ear of fruit 3 is then passed to thread 4.

2. The method for detecting the appearance quality of fresh corn ears according to claim 1, characterized in that: The fruit bunch is placed in the tray (3) and moves with the tray (3). At this time, the fruit bunch is relatively stationary relative to the tray (3). The tray (3) continues to move, and the fruit bunch comes into contact with the inclined surface of the guide rail (41). After passing the inclined surface of the guide rail (41), it floats on the guide rail (41). The inner wall on the left side of the tray (3) pushes the fruit bunch, and the fruit bunch rotates clockwise under the action of the guide rail (41).

3. The method for detecting the appearance quality of fresh corn ears according to claim 1, characterized in that: The ear of fruit was photographed a total of n times, each time from a different angle. Thread 2 processes each of the n images and counts the defects in each image.

4. The method for detecting the appearance quality of fresh corn ears according to claim 1, characterized in that: The max pooling convolutional layer includes: a convolutional structure, a batch normalization structure, a ReLU activation function, and a max pooling structure. The max pooling convolutional layer first extracts features through a 3×3 convolutional structure, then normalizes them through a batch normalization structure, then activates them through a ReLU activation function, and finally performs a 3×3 max pooling operation to obtain the dimensionality-reduced feature map.

5. The method for detecting the appearance quality of fresh corn ears according to claim 4, characterized in that: The information from several feature channels of the reduced feature map is separated into two paths: the identity mapping path and the convolutional re-extraction path. The feature information from the convolutional re-extraction path is re-extracted by convolution, while the feature information from the identity mapping path is directly mapped by identity. The two feature information paths, the identity mapping path and the convolutional re-extraction path, are concatenated by the Concat function and then shuffled by channel to obtain the feature map after feature fusion.

6. The method for detecting the appearance quality of fresh corn ears according to claim 5, characterized in that: The convolutional re-extraction path first reduces the dimensionality of the channel feature information through a 1×1 convolution, then through a 3×3 depthwise convolution, and finally through a 1×1 convolution to increase the dimensionality, while keeping the number of output feature channels unchanged.

7. The method for detecting the appearance quality of fresh corn ears according to claim 5, characterized in that: The enhanced channel feature information is fed into the first and second convolution branches respectively for feature downsampling, then concatenated by the Concat function, and then channel shuffling is performed. After downsampling, the number of output channels of the feature map is doubled and the size is halved. The first convolutional branch includes, in sequence, a 3×3 depthwise convolution and a 1×1 ordinary convolution; the second convolutional branch includes, in sequence, a 1×1 ordinary convolution, a 3×3 depthwise convolution and a 1×1 ordinary convolution.

8. The method for detecting the appearance quality of fresh corn ears according to claim 7, characterized in that: The feature map X after channel shuffling is obtained, and the size of feature map X is S×S×C1. A series of sub-feature maps X' are mapped through a lightweight convolutional layer, and the size of sub-feature map X' is S'×S'×C1. When the scaling factor is 2, there will be 4 sub-feature maps. The 4 mapped feature maps are connected along the channel dimension to obtain a feature map X” with the channel dimension increased by 4 times and the spatial dimension reduced by 2 times. Then, this feature map is input into a non-staggered convolutional layer with C2 filters. Finally, the size of the output feature map is S / 2×S / 2×C2.

Images