A corn damage automatic detection and grading system and method

Abstract

The present invention discloses a system and method for automatically detecting and grading corn damage, and relates to the technical field of corn damage detection and grading. The system comprises: a corn turning device, an image acquisition device, a conveyor belt conveying device, a corn grading device and a host computer; the conveyor belt conveying device is used to convey corn; the corn turning device is used to turn the corn 180 degrees; the image acquisition device is used to take images of the corn before and after the corn is turned 180 degrees; the host computer is used to: identify the image of the corn, determine the category of the corn, and calculate the overall damage ratio of the corn, and determine the grade of the corn according to the overall damage ratio of the corn, and control the corn grading device to sort the corn according to the category and grade of the corn. The present invention can perform non-destructive damage detection on corn (such as fresh corn) on a deep processing line, with high precision and high degree of automation, and can more accurately grade and classify corn.

Description

Technical Field

[0001] The present invention relates to the technical field of corn damage detection and grading, and in particular to a corn damage automatic detection and grading system and method. Background Art

[0002] Fresh corn has a delicious taste, rich nutritional value, and high deep processing value. It is a nutritionally balanced food that is deeply loved by consumers and is widely used in various corn deep-processed foods. In order to meet the market's growing demand for fresh corn, the planting area and output of fresh corn have grown rapidly in recent years.

[0003] Whether there is mechanical damage on the outside of fresh corn belongs to the sensory quality of fresh corn, which can directly reflect the quality of fresh corn. At present, on the fresh corn deep processing line, the quality inspection and classification of fresh corn has always been the basis of various processing, and it is also the most time-consuming and labor-intensive process. This work requires workers with solid professional skills and rich experience to carry out manual inspection and classification. While the labor cost is extremely high, the sorting effect is greatly affected by human factors, which results in the rough classification of fresh corn, unsatisfactory results, and the inability to efficiently utilize the fresh corn ears of various varieties.

[0004] In order to avoid further damage to fresh corn and improve detection accuracy and efficiency, non-contact non-destructive testing methods are gradually being applied to the field of corn detection. At present, RGB image detection and spectral image detection are widely used in the field of non-contact detection of corn. The former has fast detection speed but low accuracy. It is often used in target detection and occasions with high real-time requirements, and cannot meet the requirements of strict grading of fresh corn production lines. Hyperspectral imaging technology in spectral detection analyzes the optical properties of the target to perform non-destructive testing on the target. Compared with RGB images, hyperspectral images can not only obtain high-dimensional spectral information, but also contain spatial information of the detection target, which can reflect the internal and external quality characteristics of the target. The use of hyperspectral image information for detection and grading has the characteristics of high accuracy and non-destructiveness, which can meet the requirements of strict grading on the fresh corn processing line. Summary of the invention

[0005] The technical problem to be solved by the present invention is to address the deficiencies of the prior art and specifically provide a corn damage automatic detection and grading system and method, which are as follows:

[0006] 1) In the first aspect, the present invention provides a corn damage automatic detection and grading system, and the specific technical scheme is as follows:

[0007] It includes a corn turning device, an image acquisition device, a conveyor belt conveying device, a corn grading device and a host computer;

[0008] The conveyor belt conveyor is used to: convey corn;

[0009] The corn turning device is used to: turn the corn 180°;

[0010] The image acquisition device is used to: take images of the corn before and after the corn is turned 180 degrees;

[0011] The host computer is used to: identify the corn image, determine the corn category based on the recognition result, and calculate the overall damage ratio of the corn based on the recognition result;

[0012] The host computer is also used to determine the grade of corn according to the overall damage ratio of corn, and control the corn grading device to sort the corn according to the type and grade of corn.

[0013] The beneficial effects of the corn damage automatic detection and grading system provided by the present invention are as follows:

[0014] It can perform non-destructive damage detection on corn (such as fresh corn) on the deep processing line with high accuracy and high degree of automation, and can grade and classify corn more accurately.

[0015] Based on the above scheme, the automatic detection and grading system for corn damage of the present invention can also be improved as follows.

[0016] Furthermore, it also includes a position sensing device, which is used to: determine whether the corn has reached a preset photographing position before and after the corn is turned 180 degrees;

[0017] The image acquisition device is used to capture images of the corn before and after the corn is flipped 180 degrees and when the corn reaches a preset photographing position.

[0018] Furthermore, the corn turning device, the image acquisition device, the conveyor belt conveying device, the corn grading device and the position sensing device are all integrated on the frame;

[0019] The conveyor belt conveyor device conveys corn in a horizontal direction, and the image acquisition device captures images of the corn along a vertical downward shooting direction. The preset photographing position is located in: the image acquisition device along the vertical downward shooting direction.

[0020] Furthermore, the corn turning device includes a turning shaft and a turning lever return component, the turning shaft is rotatably arranged on the turning lever return component, a plurality of turning levers are arranged on the turning shaft, and all the turning levers are flush, a plurality of openings adapted to the turning levers are provided on the turning lever return component, and the turning shaft is connected to the servo motor through a coupling;

[0021] The upper computer is also used to control the servo motor to drive the turning shaft to rotate. When the turning shaft rotates, each turning lever is driven to rotate out of the corresponding opening to turn the corn 180 degrees.

[0022] Further, the number of image acquisition devices is 2, respectively recorded as a first image acquisition device and a second image acquisition device, the number of position sensing devices is 2, respectively recorded as a first position sensing device and a second position sensing device, and the number of preset photographing positions is 2, respectively recorded as a first preset photographing position and a second preset photographing position;

[0023] The first image acquisition device is used to: before turning the corn 180 degrees, take an image of the corn in a vertical downward shooting direction, which is recorded as a first image;

[0024] The first position sensing device is used to determine whether the corn has reached a first preset photographing position before the corn is turned 180 degrees, and the first preset photographing position is located in a vertical downward shooting direction of the first image acquisition device;

[0025] The second image acquisition device is used to: after turning the corn 180 degrees, take an image of the corn in a vertical downward shooting direction, which is recorded as a second image;

[0026] The second position sensing device is used to determine whether the corn has reached a second preset photographing position after the corn is turned 180 degrees, and the second preset photographing position is located in a vertical downward shooting direction of the second image acquisition device;

[0027] The upper computer is specifically used to: identify the first image to obtain a first recognition result, determine the first damage ratio of corn based on the first recognition result, identify the second image to obtain a second recognition result, determine the second damage ratio of corn based on the second recognition result, determine the sum of the first damage ratio and the second damage ratio as the overall damage ratio, and determine the category of corn based on the first recognition result and the second recognition result, the recognition result includes the first recognition result and the second recognition result.

[0028] Furthermore, the first image acquisition device and the second image acquisition device both include a hyperspectral camera, the first image is: a hyperspectral image of corn taken by the hyperspectral camera of the first image acquisition device, and the second image is: a hyperspectral image of corn taken by the hyperspectral camera of the second image acquisition device.

[0029] Furthermore, the host computer is also used to: use a pre-trained neural network model to recognize the first image and the second image respectively to obtain a first recognition result and a second recognition result.

[0030] Furthermore, the corn is fresh-eating corn.

[0031] 2) In the second aspect, the present invention also provides a method for automatically detecting and grading corn damage, and the specific technical scheme is as follows:

[0032] Using any of the above-mentioned corn damage automatic detection and grading systems, the method includes:

[0033] A conveyor belt conveyor transports corn;

[0034] The corn turning device turns the corn 180°;

[0035] Before and after the corn is turned 180°, the image acquisition device takes images of the corn and sends them to the host computer;

[0036] The host computer recognizes the corn image, determines the type of corn based on the recognition result, and calculates the overall damage percentage of the corn based on the recognition result;

[0037] The upper computer determines the grade of the corn according to the overall damage ratio of the corn, and controls the corn grading device to sort the corn according to the type and grade of the corn.

[0038] 3) In the third aspect, the present invention also provides a method for automatically detecting and grading corn damage, and the specific technical scheme is as follows:

[0039] The method of the corn damage automatic detection and grading system with two image acquisition devices and two position sensing devices of the present invention includes:

[0040] When the conveyor belt conveyor device conveys corn, the first position sensor device determines whether the corn reaches the first preset photographing position to obtain a first determination result;

[0041] When the first judgment result is yes, the first image acquisition device captures a first image along a vertical downward shooting direction, and sends the first image to the host computer;

[0042] When the conveyor belt conveyor device conveys the corn to the corn turning device, the corn turning device turns the corn 180 degrees;

[0043] The second position sensing device determines whether the corn has reached the second preset photographing position, and obtains a second determination result;

[0044] When the second judgment result is yes, the second image acquisition device captures a second image along a vertical downward shooting direction, and sends the second image to the host computer;

[0045] The host computer recognizes the first image and obtains a first recognition result. According to the first recognition result, the host computer determines the first damage ratio of the corn. The host computer recognizes the second image and obtains a second recognition result. According to the first recognition result, the host computer determines the second damage ratio of the corn. The sum of the first damage ratio and the second damage ratio is determined as the overall damage ratio, and the category of the corn is determined according to the first recognition result and the second recognition result.

[0046] The upper computer determines the grade of the corn according to the overall damage ratio of the corn, and controls the corn grading device to sort the corn according to the type and grade of the corn.

[0047] It should be noted that the beneficial effects achieved by the technical solutions of the second to third aspects of the present invention and the corresponding possible implementation methods can be found in the above-mentioned technical effects of the first aspect and its corresponding possible implementation methods, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments of the present invention:

[0049] Figure 1 This is a schematic diagram of the structure of a corn damage automatic detection and grading system according to an embodiment of the present invention;

[0050] Figure 2 It is a structural schematic diagram of a corn turning device;

[0051] Figure 3 is a sample image that has been manually annotated;

[0052] Figure 4 A schematic diagram of a process for automatically detecting and grading corn damage;

[0053] Figure 5 An image of the upper portion of fresh corn;

[0054] Figure 6 This is the predicted label map for the upper half of fresh corn. DETAILED DESCRIPTION

[0055] The principles and features of the present invention are described below. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.

[0056] The technical solution of the present invention and how the technical solution of the present invention solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will be described below in conjunction with the accompanying drawings.

[0057] like Figure 1 As shown, a corn damage automatic detection and grading system according to an embodiment of the present invention comprises a corn turning device 1, an image acquisition device, a conveyor belt conveying device 4, a corn grading device 6 and a host computer 10;

[0058] The conveyor belt conveying device 4 is used to: convey corn;

[0059] The corn turning device 1 is used to: turn the corn 180°;

[0060] The image acquisition device is used to: take images of the corn before and after the corn is turned 180 degrees;

[0061] The host computer 10 is used to: identify the image of corn, determine the category of corn according to the identification result, and calculate the overall damage ratio of corn according to the identification result;

[0062] The host computer 10 is also used to determine the grade of the corn according to the overall damage ratio of the corn, and control the corn grading device 6 to sort the corn according to the type and grade of the corn.

[0063] Among them, the categories of corn can be divided into: undamaged corn and damaged corn. When the recognition result only includes the intact ear area, the category of corn is: undamaged corn. When the recognition result includes any one of the abrasion area and the squeeze area, the category of corn is: damaged corn.

[0064] Among them, the grade of corn can be divided in advance according to the overall damage ratio. For example, when P=0, the grade of corn is no damage. When 0<P≤10%, the grade of corn is first-level damage. When 10<P≤20%, the grade of corn is second-level damage. The grade can also be divided according to the actual situation, where P represents the overall damage ratio.

[0065] Among them, after the upper computer 10 has finished grading the corn, the corn grading device 6 pops out a grading baffle by a cylinder to sort the corn. The number of grading baffles in the corn grading device 6 is determined according to the number of divided grades.

[0066] Optionally, the above technical solution also includes a position sensing device, which is used to determine whether the corn has reached a preset photographing position before and after the corn is flipped 180°, specifically, to determine whether the position of the corn on the conveyor belt transport device 4 has reached a preset photographing position.

[0067] The image acquisition device is used to capture images of the corn before and after the corn is flipped 180 degrees and when the corn reaches a preset photographing position.

[0068] Optionally, in the above technical solution, the corn turning device 1, the image acquisition device, the conveyor belt conveying device 4, the corn grading device 6 and the position sensing device are all integrated on the frame 5;

[0069] The conveyor belt transport device 4 transports corn in the horizontal direction, and the image acquisition device captures the image of the corn along the vertical downward shooting direction. The preset photographing position is located at: the image acquisition device along the vertical downward shooting direction.

[0070] Optionally, in the above technical solution, the corn turning device 1 includes a turning shaft 11 and a turning lever return component 12, the turning shaft 11 is rotatably arranged on the turning lever return component 12, a plurality of turning levers 13 are arranged on the turning shaft 11, and all the turning levers 13 are flush, a plurality of openings adapted to the turning levers 13 are provided on the turning lever return component 12, and the turning shaft 11 is connected to the servo motor through a coupling, specifically:

[0071] The corn turning device 1 includes a turning shaft 11, a turning lever return component 12, a turning lever 13 and a sleeve 14, wherein the turning lever 13 is installed on the turning shaft 11, and the angle between the turning shaft 11 and the turning lever return component 12 is 0° when not working. The main function of the turning device is to adjust the posture of the corn according to the command. The turning shaft 11 is connected to the external servo motor through a coupling, and the turning lever return component 12 is fixed to the frame 5. The turning lever return component 12 cooperates with the turning shaft 11 through the sleeve 14, so that the turning shaft 11 can drive the turning lever 13 to rotate. The servo motor is actuated by the upper computer 10 through the control signal pair, so that the turning lever 13 rotates around the turning shaft 11 to complete the turning of the corn. Figure 2 shown.

[0072] The upper computer 10 is also used to control the servo motor to drive the flip shaft 11 to rotate. When the flip shaft 11 rotates, each flip lever 13 is driven to rotate out from the corresponding opening to flip the corn 180 degrees.

[0073] Optionally, in the above technical solution, the number of image acquisition devices is 2, respectively recorded as a first image acquisition device 2 and a second image acquisition device 3, the number of position sensing devices is 2, respectively recorded as a first position sensing device and a second position sensing device, and the number of preset photographing positions is 2, respectively recorded as a first preset photographing position 7 and a second preset photographing position 8;

[0074] The first image acquisition device 2 is used to: before turning the corn 180 degrees, take an image of the corn in a vertical downward shooting direction, which is recorded as a first image;

[0075] The first position sensing device is used to determine whether the corn has reached the first preset photographing position 7 before the corn is turned 180 degrees. The first preset photographing position 7 is located in the vertical downward shooting direction of the first image acquisition device 2.

[0076] The second image acquisition device 3 is used to: after the corn is flipped 180 degrees, take an image of the corn in a vertical downward shooting direction, which is recorded as a second image;

[0077] The second position sensing device is used to determine whether the corn has reached the second preset photographing position 8 after the corn is turned 180 degrees. The second preset photographing position 8 is located in the vertical downward shooting direction of the second image acquisition device 3;

[0078] The upper computer 10 is specifically used to: identify the first image to obtain a first recognition result, determine the first damage ratio of corn based on the first recognition result, identify the second image to obtain a second recognition result, determine the second damage ratio of corn based on the second recognition result, determine the sum of the first damage ratio and the second damage ratio as the overall damage ratio, and determine the classification of corn based on the first recognition result and the second recognition result, the recognition result includes the first recognition result and the second recognition result.

[0079] Among them, when the first recognition result and the second recognition result only include intact ear areas, the category of corn is: undamaged corn; when the first recognition result or the second recognition result includes any one of the abrasion area and the squeezed area, the category of corn is: damaged corn.

[0080] The first recognition result and the second recognition result further include: the area of ​​the intact ear region, the area of ​​the squeezed area, and the area of ​​the abraded area. The first damage ratio can be calculated according to the area of ​​the intact ear region, the area of ​​the squeezed area, and the area of ​​the abraded area in the first recognition result, which is specifically calculated by the following formula:

[0081]

[0082] in, Indicates: the first damage ratio, Indicates: the area of ​​the abrasion area as a percentage of the first damage, Indicates: the area of ​​the crushed area accounted for by the first injury, It means: the area of ​​intact ear region is proportional to the first damage.

[0083] The second damage ratio can be calculated based on the area of ​​the intact ear region, the area of ​​the squeezed area, and the area of ​​the abraded area in the second recognition result, and is specifically calculated by the following formula:

[0084]

[0085] in, Indicates: The proportion of the second injury, Indicates: the area of ​​the abrasion area as a percentage of the second injury, Indicates: the area of ​​the crushed area accounted for by the second injury, It means: the area of ​​intact ear area is proportional to the secondary damage.

[0086] The position sensing device may be a weight sensor, that is, the first position sensor and the second position sensor are both weight sensors. When the corn reaches the preset photographing position, the pressure signal collected by the weight sensor will change under the action of the gravity of the corn. When the host computer 10 detects that the pressure signal collected by the weight sensor changes, it is determined that the corn reaches the preset photographing position. The conveyor belt conveying device 4 includes a motor 9 and a conveyor belt. The weight sensor is arranged between the conveyor belt and the platform of the frame 5. The conveyor belt rotates along the platform of the frame 5.

[0087] In another embodiment, the position sensing device includes a capacitance measuring circuit and an electrode. Specifically, the first position sensing device includes a first capacitance measuring circuit and a first electrode, and the second position sensing device includes a second capacitance measuring circuit and a second electrode. The first electrode is located in the vertically downward shooting direction of the first image acquisition device 2 and is arranged on the platform of the frame 5. The second electrode is located in the vertically downward shooting direction of the second image acquisition device 3 and is arranged on the platform of the frame 5. A metal sheet is attached to the placement position of the corn on the conveyor belt. The first capacitance measuring circuit detects the capacitance value of the capacitor composed of the first electrode in real time and sends it to the host computer 10. The first capacitance measuring circuit detects the capacitance value of the capacitor composed of the first electrode in real time and sends it to the host computer 10. When the corn reaches the first preset photographing position 7, the first electrode forms a capacitor with the metal sheet, and the first capacitance measuring circuit can measure the capacitance value. At this time, the host computer 10 determines that the corn reaches the first preset photographing position 7. When the corn reaches the second preset photographing position 8, the first electrode forms a capacitor with the metal sheet, and the second capacitance measuring circuit can measure the capacitance value. At this time, the host computer 10 determines that the corn reaches the second preset photographing position 8.

[0088] In another embodiment, the upper computer 10 uses image recognition to identify the positional relationship between the area where the corn is located and the metal sheet, and calculates the weight of the corn applied to the metal sheet through force analysis. Combined with the functional relationship between the capacitance value and the weight applied to the metal sheet calculated in advance through experiments, the weight of the corn can be reversed. The upper computer 10 uses image recognition to identify the length of the transmitted corn, which helps to control the quality of the corn.

[0089] Optionally, in the above technical solution, the first image acquisition device 2 and the second image acquisition device 3 both include hyperspectral cameras, the first image is: a hyperspectral image of corn taken by the hyperspectral camera of the first image acquisition device 2, and the second image is: a hyperspectral image of corn taken by the hyperspectral camera of the second image acquisition device 3.

[0090] Optionally, the first image acquisition device 2 and the second image acquisition device 3 both include halogen lamps, and the halogen lamps are used to provide fill light when photographing corn.

[0091] Optionally, in the above technical solution, the host computer 10 is further used to: use a pre-trained neural network model to recognize the first image and the second image respectively to obtain a first recognition result and a second recognition result.

[0092] The process of obtaining the pre-trained neural network model includes:

[0093] S101. Collect some images of fresh corn as sample images, including sample images of intact ears and ears with different types of mechanical damage. Classify and manually annotate all sample images, including three categories: undamaged ear area (i.e. intact ear area), abrasion area and squeezed area. Pixels in different areas are annotated with different category numbers and colors. For example, the pixels in the undamaged ear area are annotated as category 1, and the annotated color is blue; the pixels in the abrasion area are annotated as category 2, and the annotated color is yellow; the pixels in the squeezed area are annotated as category 3, and the annotated color is purple. Black pixels represent background areas. Figure 3 shown.

[0094] Among them, 80% of all sample images collected were set as training sets, and the remaining 20% ​​of sample images were set as test sets. The obtained image sample spatial dimension pixel size was 512×512, and the number of spectral dimension channels was 204. The area where the seed epidermis was damaged and the contents were exposed was defined as the abrasion area. The area where the epidermis was intact and the seed was wrinkled was defined as the squeezed area.

[0095] S102, the host computer 10 performs image preprocessing on each sample image to obtain multiple preprocessed sample images. The image preprocessing includes black and white correction, edge supplementation, small sample fast extraction and data normalization preprocessing to reduce the influence of illumination and solve the problem of inconsistent pixel size of sample blocks.

[0096] Among them, Python programming language is used to call scipy.io, imageio and numpy libraries for preprocessing. Black and white correction is performed to balance the brightness between different bands to improve image quality and application effects. Black and white correction can be performed using the following formula:

[0097]

[0098] In the formula, is the corrected hyperspectral image; is the original hyperspectral image; is a black calibration image obtained by turning off the light source and completely covering the lens with a lens cap; To obtain a white calibration image under the same conditions as the original image.

[0099] Among them, edge supplementation is to expand the boundary of the original image by adding a certain number of pixels around the original hyperspectral image of fresh corn to solve the problem of inconsistent pixel size of sample blocks at the edge. The specific implementation process is as follows:

[0100] First, the left edge of the hyperspectral image data is horizontally flipped and copied, and then flipped left and right to expand the width. Then, the upper edge of the hyperspectral image data is flipped upside down and copied, and then flipped up and down to expand the height.

[0101] Among them, small sample fast extraction uses a non-overlapping sampling method to perform uniform sampling in the entire image range, and sets the size of the sampling block to 3 pixels × 3 pixels. By extracting small sample blocks from the original hyperspectral image, the processing process can be made more efficient and accurate, thereby improving the performance and effect of the algorithm.

[0102] Data normalization uses the Min-Max normalization method to linearly normalize the data to between [-1, 1]. Normalization can avoid the impact of inconsistent value ranges of different features and improve feature reliability.

[0103] S103, using a plurality of pre-processed sample images to train a neural network model, performing pixel-level classification on the fresh corn images, and adjusting parameters of the neural network model to obtain various types of model accuracies under various parameters;

[0104] S104. During the test recognition, the trained neural network is called to predict the pixel-level labels of the test set after the small sample block is extracted, so as to classify the fresh corn image at the pixel level and determine whether it belongs to the undamaged area, the abraded area or the squeezed area.

[0105] When testing recognition, the trained neural network is called to predict pixel-level labels for the test set after extracting small sample blocks. This includes defining the data set reading function, defining the data loading function, defining the evaluation index function, defining the creation, defining the category color function, and loading the model.

[0106] S105 , marking and integrating the classified pre-processed sample images to form a mask image, and visually marking and saving the mask images.

[0107] S106. Calculate the area and damage ratio of each part (undamaged area, abrasion area and squeezed area), and compare them with the artificial label image (artificial standard sample image) to calculate the evaluation indicators, including overall classification accuracy, average classification accuracy and kappa coefficient, for a total of three types of classification accuracy.

[0108] S107. Save the calculation results and evaluation indexes of the area and damage ratio of each part (non-damaged area, abrasion area and squeezed area) in a table.

[0109] S108, selecting the best training model based on the evaluation index, or based on the evaluation index and the visual marking image (mask image), as the trained neural network model, specifically:

[0110] ① The process of selecting the optimal training model based on the evaluation index includes: judging whether the calculated overall classification accuracy, average classification accuracy and kappa coefficient are all greater than the corresponding preset thresholds. If so, the optimal training model is obtained; if not, the training continues.

[0111] ② Select the best training model based on evaluation indicators and visual marker images (mask images), including:

[0112] Determine whether the classification of each mark in the mask image is consistent with the actual area. If not, continue model training. If so, continue to determine whether the calculated overall classification accuracy, average classification accuracy and kappa coefficient are all greater than the corresponding preset thresholds. If so, the optimal training model is obtained. If not, continue model training.

[0113] Among them, the neural network model can be the Resnet18 network model in the torch library, or models with other network structures can be selected according to actual conditions.

[0114] Optionally, in the above technical solution, the corn is fresh corn.

[0115] An automatic corn damage detection and grading method according to an embodiment of the present invention adopts any of the above-mentioned automatic corn damage detection and grading systems, and the method comprises:

[0116] S1, the conveyor belt conveying device 4 conveys corn;

[0117] S2, the corn turning device 1 turns the corn 180°;

[0118] S3, before and after the corn is turned 180°, the image acquisition device captures the image of the corn and sends it to the host computer 10;

[0119] S4, the host computer 10 recognizes the image of the corn, determines the type of the corn according to the recognition result, and calculates the overall damage ratio of the corn according to the recognition result;

[0120] S5. The host computer 10 determines the grade of the corn according to the overall damage ratio of the corn, and controls the corn grading device 6 to sort the corn according to the type and grade of the corn.

[0121] An automatic detection and grading method for corn damage according to an embodiment of the present invention adopts an automatic detection and grading system for corn damage having two image acquisition devices and two position sensing devices according to the present invention, and the method comprises:

[0122] S201, when the conveyor belt conveying device 4 conveys corn, the first position sensing device determines whether the corn reaches the first preset photographing position, and obtains a first determination result;

[0123] S202, when the first judgment result is yes, the first image acquisition device 2 captures a first image along a vertical downward shooting direction, and sends the first image to the host computer 10;

[0124] S203, when the conveyor belt conveying device 4 conveys the corn to the corn turning device 1, the corn turning device 1 turns the corn 180 degrees;

[0125] S204, the second position sensing device determines whether the corn has reached the second preset photographing position, and obtains a second determination result;

[0126] S205, when the second judgment result is yes, the second image acquisition device 3 captures a second image along a vertical downward shooting direction, and sends the second image to the host computer 10;

[0127] S206, the upper computer 10 recognizes the first image to obtain a first recognition result, determines a first damage ratio of corn based on the first recognition result, recognizes the second image to obtain a second recognition result, determines a second damage ratio of corn based on the first recognition result, determines the sum of the first damage ratio and the second damage ratio as the overall damage ratio, and determines the category of corn based on the first recognition result and the second recognition result.

[0128] S207, the host computer 10 determines the grade of the corn according to the overall damage ratio of the corn, and controls the corn grading device 6 to sort the corn according to the type and grade of the corn.

[0129] In another embodiment of a method for automatically detecting and grading corn damage of the present invention, the automatic detection and grading system for corn damage adopted comprises a corn turning device 1, a conveyor belt conveying device 4, a frame 5, a host computer 10, a corn grading device 6 and two image acquisition devices (a first image acquisition device 2 and a second image acquisition device 3) and two position sensing devices (a first position sensing device and a second position sensing device), wherein the corn turning device 1 is used to turn the fresh corn 180°; the first image acquisition device 2 and the second image acquisition device 3 comprise a halogen lamp and a hyperspectral camera for acquiring images of the fresh corn; the conveyor belt conveying device 4 comprises a motor 9 and a conveyor belt for conveying the fresh corn from one position to another; the frame 5 is the corn turning device 1, the conveyor belt conveying device 4, the corn grading device 6 and the two image acquisition devices (a first image acquisition device 2 and a second image acquisition device 3) and the two position sensing devices (a first position sensing device and a second position sensing device). 6. The first image acquisition device 2, the second image acquisition device 3, the first position sensor device and the second position sensor device provide a fixed position for installation; the host computer 10 is used to control the corn turning device 1, the conveyor belt conveying device 4, the corn grading device 6, the first image acquisition device 2, the second image acquisition device 3, the first position sensor device and the second position sensor device, and the host computer 10 is used to pre-train the neural network model, and use the trained neural network model to classify the collected images of fresh corn and calculate the overall damage ratio, and finally classify the fresh corn; the corn grading device 6 classifies the fresh corn that has been detected and sends it to the corn category of the corresponding grade; the first position sensor device and the second position sensor device are used to determine whether the fresh corn has reached the corresponding preset photo position, such as Figure 4 As shown, the method includes:

[0130] S301, the corn turning device 1, the conveyor belt conveying device 4, the corn grading device 6, the first image acquisition device 2, the second image acquisition device 3 and the first position sensor device and the second position sensor device are installed on the frame 5, and after being debugged to normal operation in conjunction with the host computer 10, the host computer 10 controls the motor 9 of the conveyor belt conveying device 4 to drive the conveyor belt to transport fresh corn.

[0131] S302, the first position sensing device detects whether the fresh corn reaches the first preset photographing position 7. If not, no other operation is performed, and the host computer 10 controls the motor 9 of the conveyor belt conveying device 4 to continue driving the conveyor belt to transport the fresh corn. If yes, the host computer 10 controls the motor 9 of the conveyor belt conveying device 4 to stop driving the conveyor belt so that the fresh corn stays at the first preset photographing position 7.

[0132] S303. Under the illumination of the halogen lamp of the first image acquisition device 2, the hyperspectral camera of the first image acquisition device 2 captures an image of the upper half of the fresh corn and transmits it to the host computer 10. The image of the upper half of the fresh corn is the first image, that is, before the fresh corn is flipped 180°, the hyperspectral camera of the first image acquisition device 2 captures the image of the fresh corn in the vertical downward shooting direction. The upper half of the fresh corn refers to: before the fresh corn is flipped 180°, the hyperspectral camera of the first image acquisition device 2 captures the local area of ​​the fresh corn in the vertical downward shooting direction. The first image is a hyperspectral image. The image of the upper half of the fresh corn is as shown in FIG. Figure 5 shown.

[0133] S304, after the host computer 10 obtains the first image, the host computer 10 controls the motor 9 of the conveyor belt conveying device 4 to continue to drive the conveyor belt to transport the fresh corn, and the host computer 10 uses the pre-trained neural network model to recognize the obtained first image, obtains the first recognition result, and calculates the damage ratio of the upper half of the fresh corn, that is, the first damage ratio, and saves it, and saves the corresponding prediction label map, such as Figure 6 shown.

[0134] S305, when the conveyor belt conveys the fresh corn to the corn turning device 1, the corn turning device 1 is started to turn the fresh corn 180 degrees, and the upper computer 10 controls the motor 9 of the conveyor belt conveying device 4 to continue to drive the conveyor belt to transport the fresh corn.

[0135] Among them, under normal conditions, the ends of the arranged and flush flip levers 13 are set close to the conveyor belt. According to the conveyor belt speed and the length of the fresh corn conveyed, the cycle of lifting the flip lever 13 can be calculated. The flip lever 13 can be controlled to be lifted according to the cycle to flip the fresh corn 180°.

[0136] S306, the second position sensing device detects whether the fresh corn reaches the second preset photographing position 8. If not, no other operation is performed, and the host computer 10 controls the motor 9 of the conveyor belt conveying device 4 to continue driving the conveyor belt to transport the fresh corn. If yes, the host computer 10 controls the motor 9 of the conveyor belt conveying device 4 to stop driving the conveyor belt so that the fresh corn stays at the second preset photographing position 8.

[0137] S307. Under the illumination of the halogen lamp of the second image acquisition device 3, the hyperspectral camera of the second image acquisition device 3 captures an image of the lower half of the fresh corn and transmits it to the host computer 10. The image of the lower half of the fresh corn is the second image, that is, the image of the fresh corn captured by the hyperspectral camera of the second image acquisition device 3 along the vertical downward shooting direction after the fresh corn is flipped 180°. The lower half of the fresh corn refers to a local area of ​​the fresh corn captured by the hyperspectral camera of the second image acquisition device 3 along the vertical downward shooting direction after the fresh corn is flipped 180°. The second image is a hyperspectral image.

[0138] S308. After the host computer 10 obtains the second image, the host computer 10 controls the motor 9 of the conveyor belt transport device 4 to continue driving the conveyor belt to transport the fresh corn, and the host computer 10 uses the pre-trained neural network model to recognize the obtained second image, obtains the second recognition result, and calculates the damage ratio of the lower half of the fresh corn, i.e., the second damage ratio, and saves it.

[0139] S309, the host computer 10 determines whether the first damage ratio and the second damage ratio are obtained. If there is only the first damage ratio or the second damage ratio, or there is no first damage ratio or the second damage ratio, the host computer 10 drives the motor 9 of the conveyor belt conveying device 4 to drive the conveyor belt to transport the fresh corn back to the first preset shooting position and / or the second preset shooting position, and re-takes pictures and identifies until the first damage ratio and the second damage ratio are obtained, and then calculates the overall damage ratio of the fresh corn, and determines the category of the fresh corn according to the first recognition result and the second recognition result.

[0140] Among them, when the upper computer 10 drives the motor 9 of the conveyor belt conveying device 4 to drive the conveyor belt to transport fresh corn back to the first preset shooting position, the upper computer 10 controls the lifting of the arranged and flush flip lever 13 to make the conveyor belt transport fresh corn return to the first preset shooting position.

[0141] S310, the host computer 10 determines the grade of the fresh corn according to the overall damage ratio of the fresh corn, and controls the corn grading device 6 to sort the fresh corn according to the type and grade of the fresh corn, and starts the conveyor belt to convey the fresh corn to the corresponding storage location.

[0142] The beneficial effects of the present invention are as follows:

[0143] 1) The deep learning-based method of the present invention can more accurately determine the overall damage ratio of fresh corn when detecting deep processing of fresh corn. The neural network model is trained in advance, and the first damage ratio, the second damage ratio and the visual prediction label map of each part can be directly output through image preprocessing and Resnet18 network.

[0144] 2) Compared with manual grading, the present invention is more accurate and will not cause secondary damage to fresh corn, which has important reference significance for improving the processing efficiency and quality of fresh corn.

[0145] In the above embodiments, although the steps are numbered S1, S2, etc., these are only specific embodiments given by the present invention. Those skilled in the art may adjust the execution order of S1, S2, etc. according to actual conditions, which is also within the protection scope of the present invention. It can be understood that in some embodiments, some or all of the above embodiments may be included.

[0146] It should be noted that the beneficial effects of the method for automatically detecting and grading corn damage provided in the above embodiment are the same as the beneficial effects of the system for automatically detecting and grading corn damage provided in the above embodiment, which will not be described in detail here. The method and system embodiments provided in the above embodiment belong to the same concept, and their specific implementation process is detailed in the method embodiment, which will not be described in detail here.

[0147] The above description is only a preferred embodiment of the present invention and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present invention is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present invention (but not limited to) to form a technical solution.

[0148] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects and represent the definition of a specific order or sequence. The order of use of similar objects can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than the order shown or described.

[0149] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A corn damage automatic detection and grading system, characterized in that: It includes a corn turning device, an image acquisition device, a conveyor belt conveying device, a corn grading device and a host computer; The conveyor belt conveying device is used to: convey corn; The corn turning device is used to: turn the corn 180 degrees; The image acquisition device is used to: take images of the corn before and after the corn is flipped 180 degrees; The host computer is used to: identify the image of the corn, determine the category of the corn according to the identification result, and calculate the overall damage ratio of the corn according to the identification result; The host computer is also used to: determine the grade of the corn according to the overall damage ratio of the corn, and control the corn grading device to sort the corn according to the type and grade of the corn; It also includes a position sensor device, which is used to: determine whether the corn reaches a preset photographing position before and after the corn is turned 180 degrees; The image acquisition device is used to: take an image of the corn before and after the corn is turned 180 degrees and when the corn reaches the preset photographing position; The corn turning device, the image acquisition device, the conveyor belt conveying device, the corn grading device and the position sensing device are all integrated on the frame; The conveyor belt conveyor device conveys the corn in a horizontal direction, the image acquisition device captures the image of the corn in a vertically downward shooting direction, and the preset photographing position is located in the vertically downward shooting direction of the image acquisition device; The corn turning device comprises a turning shaft and a turning lever return component, wherein the turning shaft is rotatably arranged on the turning lever return component, a plurality of turning levers are arranged on the turning shaft, and all the turning levers are flush, a plurality of openings adapted to the turning levers are provided on the turning lever return component, and the turning shaft is connected to a servo motor through a coupling; The host computer is also used to: control the servo motor to drive the flip shaft to rotate, and when the flip shaft rotates, each flip lever is driven to rotate out of the corresponding opening to flip the corn 180 degrees; The position sensing device includes a capacitance measuring circuit and an electrode. The electrode is located in the vertical downward shooting direction of the image acquisition device and is arranged on the platform of the frame. A metal sheet is attached to the placement position of the corn on the conveyor belt. When the corn reaches the preset shooting position, the first electrode and the metal sheet form a capacitor. The capacitance measuring circuit detects the capacitance value of the capacitor formed by the electrode in real time and sends it to the host computer. When the capacitance value is detected, the host computer determines that the corn has reached the preset shooting position. The host computer uses image recognition to identify the positional relationship between the area where the corn is located and the metal sheet, and calculates the weight of the corn applied to the metal sheet through force analysis. Combined with the functional relationship between the capacitance value and the weight applied to the metal sheet calculated in advance through experiments, the weight of the corn is deduced.

2. The corn damage automatic detection and grading system according to claim 1, characterized in that: The number of the image acquisition devices is 2, which are respectively recorded as a first image acquisition device and a second image acquisition device; the number of the position sensing devices is 2, which are respectively recorded as a first position sensing device and a second position sensing device; the number of the preset photographing positions is 2, which are respectively recorded as a first preset photographing position and a second preset photographing position; The first image acquisition device is used to: before turning the corn 180 degrees, take an image of the corn in a vertical downward shooting direction, which is recorded as a first image; The first position sensing device is used to determine whether the corn has reached the first preset photographing position before the corn is turned 180 degrees, and the first preset photographing position is located in the vertical downward shooting direction of the first image acquisition device; The second image acquisition device is used to: after turning the corn by 180 degrees, take an image of the corn in a vertical downward shooting direction, which is recorded as a second image; The second position sensing device is used to determine whether the corn has reached the second preset photographing position after the corn is turned 180 degrees, and the second preset photographing position is located in the vertical downward shooting direction of the second image acquisition device; The host computer is specifically used to: identify the first image to obtain a first recognition result, determine a first damage ratio of the corn based on the first recognition result, identify the second image to obtain a second recognition result, determine a second damage ratio of the corn based on the second recognition result, determine the sum of the first damage ratio and the second damage ratio as the overall damage ratio, and determine the category of the corn based on the first recognition result and the second recognition result, and the recognition result includes the first recognition result and the second recognition result.

3. The corn damage automatic detection and grading system according to claim 2, characterized in that: The first image acquisition device and the second image acquisition device both include a hyperspectral camera. The first image is a hyperspectral image of corn taken by the hyperspectral camera of the first image acquisition device. The second image is a hyperspectral image of corn taken by the hyperspectral camera of the second image acquisition device.

4. The automatic detection and grading system for corn damage according to claim 3, characterized in that: The host computer is also used to: use a pre-trained neural network model to recognize the first image and the second image respectively, to obtain the first recognition result and the second recognition result.

5. A corn damage automatic detection and grading system according to any one of claims 1 to 4, characterized in that: The corn is fresh-eating corn.

6. A method for automatically detecting and grading corn damage, characterized in that: Using the corn damage automatic detection and grading system according to any one of claims 1 to 5, the method comprises: The conveyor belt conveyor device conveys corn; The corn turning device turns the corn 180 degrees; Before and after the corn is flipped 180°, the image acquisition device takes an image of the corn and sends it to the host computer; The host computer recognizes the image of the corn, determines the category of the corn according to the recognition result, and calculates the overall damage ratio of the corn according to the recognition result; The host computer determines the grade of the corn according to the overall damage ratio of the corn, and controls the corn grading device to sort the corn according to the type and grade of the corn.

7. A method for automatically detecting and grading corn damage, characterized in that: The method of using the corn damage automatic detection and grading system according to claim 3 or 4 comprises: When the conveyor belt conveyor device conveys corn, the first position sensor device determines whether the corn reaches the first preset photographing position to obtain a first determination result; When the first judgment result is yes, the first image acquisition device captures the first image along a vertical downward shooting direction, and sends the first image to the host computer; When the conveyor belt conveying device conveys the corn to the corn turning device, the corn turning device turns the corn 180 degrees; The second position sensing device determines whether the corn has reached the second preset photographing position to obtain a second determination result; When the second judgment result is yes, the second image acquisition device captures the second image along a vertical downward shooting direction, and sends the second image to the host computer; The host computer recognizes the first image to obtain the first recognition result, determines the first damage ratio of the corn according to the first recognition result, recognizes the second image to obtain the second recognition result, determines the second damage ratio of the corn according to the first recognition result, determines the sum of the first damage ratio and the second damage ratio as the overall damage ratio, and determines the category of the corn according to the first recognition result and the second recognition result; The host computer determines the grade of the corn according to the overall damage ratio of the corn, and controls the corn grading device to sort the corn according to the type and grade of the corn.

Images