A corn mildew degree online detection system and method based on semantic segmentation

Abstract

The application discloses to the technical field of agricultural material detection, and particularly relates to a corn mildew degree online detection system and method based on semantic segmentation. The system comprises a material conveying mechanism, a material rotating mechanism, a control mechanism and an image acquisition and detection mechanism; the material conveying mechanism is used for conveying the material to be detected; the material rotating mechanism is used for controlling the material to be detected to rotate one round; the control mechanism is used for detecting whether the material to be detected reaches a detection station and controlling the material conveying mechanism and the material rotating mechanism to switch the working state; the image acquisition and detection mechanism is used for acquiring an image of the material to be detected at intervals of 120 degrees when the material to be detected rotates one round, inputting the three acquired images of the material to be detected into a corn image semantic segmentation network model to perform online detection of the corn mildew degree, and calculating the classification of the material to be detected. The problems of large labor quantity, long time consumption and low efficiency in manual operation are solved, and the application has good social and economic benefits.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of agricultural material detection, and particularly relates to a corn mildew degree online detection system and method based on semantic segmentation. BACKGROUND

[0002] Corn crops are introduced into China and widely planted due to their strong adaptability, simple planting method, high yield, and high quality, and the planting regions are mainly distributed in the southwest, north China, and northeast China, and the yield ranks first in China for several consecutive years. Corn is not only an important food crop for human beings, but also is widely used in the production of starch, fermentation processing, oil extraction, animal energy feed, and other aspects, and has a wide range of uses, which is of great significance to food security and production and life. However, due to the complex planting environment of corn, the weather is rainy and humid during the harvesting season, which can cause part of the corn to mildew and deteriorate, and the hidden deterioration occurs before harvesting. Since the corn embryo accounts for one-third of the whole grain volume, it is rich in nutrients and has a sweet taste, and the soluble sugar content is large, the respiration intensity is large, the hygroscopicity is strong, the bacterial load is large, and the original moisture of newly harvested corn is large, which is difficult to dry, so that the physiological activity of corn is strong, and in a closed environment, respiration can cause the storage temperature to rise, and under high humidity conditions, mold can grow rapidly, so it is easy to cause secondary mycotoxin pollution in the processes of airing, storage, and transportation. Corn contaminated by mycotoxins can cause serious harm, so the detection of corn mildew is particularly important. The existing corn mildew detection method is mainly manual screening, and the mechanical and automatic levels of corn screening operation are low. Therefore, an online corn mildew degree detection system and method based on semantic segmentation are needed to solve the problems of large labor intensity, long time consumption, and low efficiency of manual operation. SUMMARY

[0003] The purpose of the present application is to provide an online corn mildew degree detection system based on semantic segmentation, which is characterized by comprising: a material conveying mechanism, a material rotating mechanism, a control mechanism, and an image acquisition and detection mechanism; the material conveying mechanism comprises: a rack and a conveying driving mechanism, the conveying driving mechanism comprises a first stepper motor; the material rotating mechanism comprises: a transmission track and a rotating driving mechanism, the rotating driving mechanism comprises a second stepper motor; the control mechanism comprises: an embedded controller and a pair of photoelectric switches; the image acquisition and detection mechanism comprises: a detection computing device based on semantic segmentation, a camera support, and an industrial camera.

[0004] The conveying driving mechanism is arranged on the top of the frame, two side walls of the frame are arranged in parallel with two packaging side plates, a rotating driving mechanism is fixed between the two packaging side plates, the rotating driving mechanism drives the movement of the transmission track, the upper surface of the transmission track is in contact with the lower surface of the conveying driving mechanism, a camera support is arranged along the center line of the top of the frame in the orthographic projection of the upper surface of the transmission track, the center line is perpendicular to the moving direction of the material to be detected, an industrial camera is arranged at the center of the top of the camera support, a pair of light-receiving photoelectric switches are arranged at the two intersection points of the center line and the top of the frame, the pair of light-receiving photoelectric switches and the industrial camera form a detection plane, the detection plane is perpendicular to the upper surface of the conveying driving mechanism, a first stepper motor, a second stepper motor and the pair of light-receiving photoelectric switches are connected with an embedded controller respectively, and a detection calculation device based on semantic segmentation is connected with the embedded controller and the industrial camera respectively.

[0005] The material conveying mechanism is used for conveying the material to be detected, the material rotating mechanism is used for controlling the material to be detected to rotate one round, the control mechanism is used for detecting whether the material to be detected reaches the detection station and controlling the material conveying mechanism and the material rotating mechanism to switch the working state, and the image acquisition and detection mechanism is used for acquiring an image of the material to be detected at intervals of 120° when the material to be detected rotates one round, inputting three acquired images of the material to be detected into a corn image semantic segmentation network model to detect the degree of corn mildew online, and calculating the classification of the material to be detected.

[0006] The conveying driving mechanism further comprises a first synchronous pulley, a first synchronous belt, a first transmission sprocket, a second transmission sprocket, a first transmission chain, a third transmission sprocket, a fourth transmission sprocket, a second transmission chain and a chain tray, the first stepper motor drives the first synchronous pulley to rotate, the first synchronous pulley drives the fourth synchronous pulley through the first synchronous belt, the fourth synchronous pulley drives the first transmission sprocket to rotate, the first transmission sprocket drives the second transmission sprocket to rotate through the first transmission chain, the first transmission sprocket is coaxial with the third transmission sprocket, the third transmission sprocket drives the fourth transmission sprocket to rotate through the second transmission chain, a plurality of roller mounting seats are arranged at equal intervals on the outer surfaces of the first transmission chain and the second transmission chain arranged in parallel, the ends of each rotating roller are embedded with bearing terminals, and the bearing terminals are fixedly connected with the roller mounting seats in the opposite positions, the distance between the adjacent two rotating rollers is less than the width of a single material to be detected, and the chain tray is used for supporting the first transmission chain and the second transmission chain.

[0007] The rotating drive mechanism further comprises a second synchronous pulley, a second synchronous belt, a third synchronous pulley, a first tension roller shaft, a second tension roller shaft, and a track tray; the second step motor drives the second synchronous pulley to rotate; the second synchronous pulley drives the third synchronous pulley to rotate through the second synchronous belt; the third synchronous pulley is coupled with the first tension roller shaft and drives the first tension roller shaft to rotate; the first tension roller shaft drives the second tension roller shaft to rotate through the transmission track; and the track tray is used to support the transmission track so that the upper surface of the transmission track is in contact with the lower surface of the rotating roller.

[0008] The control mechanism further comprises a first step driver and a second step driver; the pair of photoelectric switches comprise a first pair of photoelectric switches and a second pair of photoelectric switches; the first step motor is connected to the embedded controller through the first step driver, the second step motor is connected to the embedded controller through the second step driver, and the first pair of photoelectric switches and the second pair of photoelectric switches are connected to the embedded controller.

[0009] One end of the material conveying mechanism is provided with a discharging mechanism for realizing vertical falling of a single to-be-tested material through the bottom opening of the discharging hopper to between the adjacent two rotating rollers of the material conveying mechanism, and ensuring that the next to-be-tested material cannot fall when the material conveying mechanism stops moving; the other end of the material conveying mechanism is provided with a discharging buffer tank for storing the to-be-tested material after detection.

[0010] The discharging mechanism comprises a discharging hopper and a hopper support, and the width of the hopper support is greater than the width of the material conveying mechanism; the bottom of the discharging hopper vertically penetrates the opening on the upper surface of the hopper support; the discharging hopper is a reverse trapezoidal hopper; the bottom of the discharging hopper is a rectangular opening; and the distance between the bottom opening of the discharging hopper and the rotating roller is 50 mm.

[0011] The image acquisition and detection mechanism is further provided with a scanning gun connected to the detection computing device based on semantic segmentation, which realizes scanning of the batch identification code of the to-be-tested material, and is used to confirm the type and batch of the to-be-tested material.

[0012] The image acquisition and detection mechanism is further provided with a display device connected to the detection computing device based on semantic segmentation, which realizes real-time output of the detection result, and the detection result comprises the batch, quantity, corn mildewing degree, corn kernel missing degree, and classification of the to-be-tested material.

[0013] The classification of the to-be-tested material is:

[0014] When the classification parameter K satisfies 0≤K<0.1, the classification of the to-be-tested material is defined as level I;

[0015] When the classification parameter K satisfies 0.1≤K<0.3, the classification of the to-be-tested material is defined as level II;

[0016] When the grading parameter K satisfies 0.3≤K<0.5, the material to be tested is defined as Grade III;

[0017] When the grading parameter K satisfies 0.5≤K≤1.0, the material to be tested is defined as Grade IV;

[0018] The grading parameter K=Q+M / 2;

[0019] In the formula, Q is the degree of kernel deficiency of the corn, and M is the degree of mold of the corn, both Q and M being percentages.

[0020] The corn image semantic segmentation network model comprises a feature extraction part and an up-sampling part, the feature extraction part comprising two convolution layers and one maximum pooling layer, and the up-sampling part comprising one feature fusion layer and two convolution layers;

[0021] The convolution layer uses a 3x3 convolution kernel and does not include a padding operation, and is used to extract local features of the image;

[0022] The maximum pooling layer uses a 2x2 window with a step of 2, so that the feature map size is halved, but the most important feature information is retained;

[0023] The feature fusion layer fuses the feature layer obtained by convolution and the feature layer obtained by up-sampling, and gradually restores the spatial dimension and detail information of the image;

[0024] A CBAM attention mechanism is added between the convolution layer and the feature fusion layer;

[0025] The input of the corn image semantic segmentation network model is a surface image of the material to be tested;

[0026] The output of the corn image semantic segmentation network model is a detection result image obtained by segmenting the input surface image of the material to be tested; the detection result image uses different colors to represent the mold area, the kernel deficiency area, the healthy area and the irrelevant background area;

[0027] The surface image of the material to be tested is obtained by splicing three images of the material to be tested collected at intervals of 120°;

[0028] The size of the surface image of the material to be tested is 512x512, the unit is pixel, and the color channel is 3 channels;

[0029] The comprehensive data set used by the corn image semantic segmentation network model comprises: an original data set and an enhanced data set; the original data set comprises a plurality of surface images of the to-be-tested material; the comprehensive data set is an enhanced image obtained by performing enhancement processing on the surface images of the to-be-tested material in the original data set; the enhancement processing comprises: randomly performing a noise adding operation and a flipping operation on the original surface images of the to-be-tested material; the noise adding operation is to randomly add salt and pepper noise; and the flipping operation is to randomly rotate by 45 degrees or 90 degrees;

[0030] The number of pictures in the comprehensive data set is greater than 500, and the comprehensive data set is divided into a training set and a test set, and the ratio of the training set to the test set is 9:1.

[0031] The corn image semantic segmentation network model uses the training set for feature extraction; an Adam optimizer is selected, an initial learning rate is defined to be less than or equal to 0.0001, a learning decay rate is defined to be 0.5 times the original learning rate, and the number of iterations is set to 200.

[0032] Another object of the present application is a detection method of the online detection system for the degree of corn mildew based on semantic segmentation according to the present application, characterized in that it comprises the following steps:

[0033] The to-be-tested material falls into the space between the adjacent first rotating roller 809 and the second rotating roller 810 on the material conveying mechanism in turn;

[0034] The control mechanism controls the rotation of the first stepping motor 704, the first stepping motor 704 drives a group of transmission sprockets 706 to rotate through the first synchronous pulley 705, and the first transmission chain 701 drives the rotating rollers to move at a uniform speed in turn; the to-be-tested material is lifted by the adjacent first rotating roller 809 and the second rotating roller 810 to realize horizontal movement;

[0035] When the to-be-tested material passes through the detection plane, the first pair of light barriers and the second pair of light barriers are triggered, the control mechanism judges that the to-be-tested material reaches the detection station, and the first stepping motor stops rotating and the second stepping motor starts rotating, the track tensioning roller shaft is driven to move under the action of the second synchronous pulley, the transmission track is driven to rotate under the action of the friction force, and the first rotating roller and the second rotating roller rotate along the axial direction under the action of the friction force between the upper surface of the transmission track and the lower surfaces of the first rotating roller and the second rotating roller, and the to-be-tested material starts to rotate 360 degrees along the axial direction under the action of the friction force, and the image acquisition and detection mechanism shoots a surface image of the to-be-tested material every 120 degrees in the process, and a total of three surface images of the to-be-tested material are shot in the process of rotating 360 degrees;

[0036] The surface image of the to-be-tested material is input into a corn image semantic segmentation network model for processing, three surface images of the to-be-tested material are merged into one merged picture with a size of 512*512, a unit of pixels and 3 channels of color channels, the merged picture is segmented to obtain a detection result image; the detection result image uses different colors to represent the moldy area, the kernel missing area, the healthy area and the irrelevant background area; the image acquisition and detection mechanism calculates the corn kernel missing degree Q and the corn mold degree M based on the detection result image, calculates the grading parameter K based on the corn kernel missing degree Q and the corn mold degree M, and calculates the grading of the to-be-tested material based on the grading parameter K.

[0037] The image acquisition and detection mechanism sends a signal to the control mechanism, the control mechanism controls the second stepper motor 803 to stop rotating, and the first stepper motor 704 continues to rotate to move the to-be-tested material out of the detection station until the next to-be-tested material triggers the first pair of photoelectric switches 507 and the second pair of photoelectric switches 510 again, and the cycle continues until all to-be-tested materials are detected.

[0038] The present application has the following advantages:

[0039] The present application discloses a corn mold degree online detection system and method based on semantic segmentation. The detection system comprises a packaging side plate, a feeding hopper, a hopper support, a synchronous belt pulley, two groups of transmission chain wheels, a transmission chain, a first stepper motor, a second stepper motor, an industrial camera, a camera support, a light supplementing lamp strip, two stepper drivers, an Arduino single-chip microcomputer and a feeding buffer box. The system realizes the automation of corn mold degree detection, has technical advantages of high efficiency and accuracy, and has good social and economic benefits.

[0040] The present application discloses a corn image semantic segmentation network model, which realizes pixel-level segmentation of mold, healthy and kernel missing areas of input corn images, can obtain higher segmentation accuracy, and outputs clear and clear images after detection, wherein dark green represents the mold area, light yellow represents the kernel missing area, dark yellow represents the healthy area, and black represents the irrelevant background. Compared with the prior art which only segments out the defect area, the present model processes the input picture, removes redundant background information, makes the detection result image clearer and clearer, and outputs and saves the kernel missing degree of corn. The present model detects corn more accurately, and the classification index and basis are more comprehensive.

[0041] The corn image semantic segmentation network model disclosed by the application has the characteristics of high efficiency and simple construction, and good results can be obtained without a large amount of data set training. The front half of the detection model of the application is feature extraction, and the latter half is up sampling. The feature extraction part is composed of convolution operation and down sampling operation, and the convolution structure used is uniformly 3x3 convolution kernel. The up sampling part is used to restore the original resolution of the feature map, and is commonly realized by transposed convolution and interpolation. In the process of model training, high gradient descent is used, so that the subsequent learning rate of the trained model is in a self-regulating process.

[0042] The detection method disclosed by the application solves the problems of labor waste and inaccurate judgment in artificial detection by using semantic segmentation technology to detect the moldy degree of corn instead of manual detection. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 The application discloses a corn moldy degree online detection system structure diagram based on semantic segmentation;

[0044] Figure 2 The application discloses a corn moldy degree online detection system structure diagram based on semantic segmentation;

[0045] Figure 3 The application discloses a corn moldy degree online detection system structure diagram based on semantic segmentation;

[0046] Figure 4 The application discloses a corn moldy degree online detection system structure diagram based on semantic segmentation;

[0047] Figure 5 The application discloses a corn moldy degree online detection system structure diagram based on semantic segmentation;

[0048] Figure 6 The application discloses a corn moldy degree online detection system structure diagram based on semantic segmentation;

[0049] Figure 7 The application discloses a corn moldy degree online detection system structure diagram based on semantic segmentation;

[0050] Figure 8 The application discloses a corn moldy degree online detection system structure diagram based on semantic segmentation;

[0051] Figure 9 The application discloses a corn moldy degree online detection system structure diagram based on semantic segmentation;

[0052] Figure 10 The application discloses a corn moldy degree online detection system structure diagram based on semantic segmentation;

[0053] Figure 11 The application discloses a corn moldy degree online detection system structure diagram based on semantic segmentation;

[0054] Figure 12 A corn image semantic segmentation network model structure diagram for the embodiment;

[0055] Figure 13 For Figure 12 An enlarged structure diagram of the CBAM model in the embodiment;

[0056] Among them: 501-industrial camera, 502-feeding hopper, 503-hopper support, 504-encapsulation side plate, 505-rotary carrier roller, 506-photoelectric switch mounting base, 507-first pair of photoelectric switches, 508-feeding buffer tank, 509-transmission track, 510-second pair of photoelectric switches, 511-camera support, 512-light supplementing lamp strip, 601-fourth synchronous pulley, 701-first transmission chain, 702-carrier roller mounting seat, 703-first synchronous belt, 704-first stepper motor, 705-first synchronous pulley, 706-first transmission sprocket, 707-chain tray, 708-track tray, 710-second transmission sprocket, 711-bearing terminal, 801-first tension roller shaft, 802-second synchronous belt, 803-second stepper motor, 804-second synchronous pulley, 805-third synchronous pulley, 808-second tension roller shaft, 809-first rotary carrier roller, 810-second rotary carrier roller, 901-first light supplementing lamp strip fixing piece, 907-second light supplementing lamp strip fixing piece. DETAILED DESCRIPTION

[0057] The application provides a corn mildew degree online detection system and method based on semantic segmentation, which will be further described in detail below with reference to the drawings.

[0058] As Figure 1 The embodiment of the application discloses a corn mildew degree online detection system based on semantic segmentation, which comprises a material conveying mechanism, a material rotating mechanism, a control mechanism and an image acquisition and detection mechanism.

[0059] As Figure 2 The material conveying mechanism is used for conveying the material to be detected; the material rotating mechanism is used for controlling the material to be detected to rotate one circle; the control mechanism is used for detecting whether the material to be detected reaches a detection station and controlling the material conveying mechanism and the material rotating mechanism to switch the working state; and the image acquisition and detection mechanism is used for acquiring one image of the material to be detected at intervals of 120 degrees when the material to be detected rotates one circle, inputting the three acquired images of the material to be detected into a corn image semantic segmentation network model to perform online detection on the corn mildew degree, and calculating the classification of the material to be detected.

[0060] In the embodiment, the material to be detected is corn, and the corn is in a cylindrical axisymmetric shape, and similar shapes also include carrots, cucumbers, long eggplants and sweet potatoes.

[0061] In the embodiment, the material conveying mechanism is used for conveying the material to be tested;

[0062] The material conveying mechanism comprises a rack and a conveying driving mechanism, and the conveying driving mechanism comprises a first stepper motor 704;

[0063] The conveying driving mechanism further comprises a first synchronous pulley 705, a first synchronous belt 703, a first transmission sprocket 706, a second transmission sprocket 710, a first transmission chain 701, a third transmission sprocket, a fourth transmission sprocket, a second transmission chain and a chain tray 707; the first stepper motor 704 drives the first synchronous pulley 705 to rotate, the first synchronous pulley 705 drives the fourth synchronous pulley 601 through the first synchronous belt 703, and the fourth synchronous pulley 601 drives the first transmission sprocket 706 to rotate; the first transmission sprocket 706 drives the second transmission sprocket 710 to rotate through the first transmission chain 701; the first transmission sprocket 706 is coaxial with the third transmission sprocket, and the third transmission sprocket drives the fourth transmission sprocket to rotate through the second transmission chain; a plurality of roller mounting seats 702 are arranged at equal intervals on the outer surfaces of the first transmission chain 701 and the second transmission chain arranged in parallel, and the ends of each rotating roller are embedded in bearing terminals 711, and the bearing terminals 711 are fixedly connected with the roller mounting seats 702 in the opposite positions; the distance between the adjacent two rotating rollers is less than the width of a single material to be tested; and the chain tray 707 is used for supporting the first transmission chain 701 and the second transmission chain.

[0064] In the embodiment, as shown in Figure 7 The material conveying mechanism is composed of a first stepper motor, a synchronous pulley, two groups of transmission sprockets, two parallel transmission chains and 33 rotating rollers arranged at equal intervals between the two transmission chains, and adjacent two rotating rollers support a corn.

[0065] In the embodiment, the chain tray 707 is located directly below the chains, which helps the lower surface of the rotating roller to be close to the upper surface of the transmission track 509.

[0066] In the embodiment, the material rotating mechanism is used for controlling the material to be tested to rotate one circle;

[0067] The material rotating mechanism comprises a transmission track 509 and a rotating driving mechanism, and the rotating driving mechanism comprises a second stepper motor 803;

[0068] The rotating driving mechanism further comprises a second synchronous pulley 804, a second synchronous belt 802, a third synchronous pulley 805, a first tension roller shaft 801, a second tension roller shaft 808, and a track tray 708; the second step motor 803 drives the second synchronous pulley 804 to rotate; the second synchronous pulley 804 drives the third synchronous pulley 805 to rotate through the second synchronous belt 802; the third synchronous pulley 805 is coupled with the first tension roller shaft 801 and drives the first tension roller shaft 801 to rotate; the first tension roller shaft 801 drives the second tension roller shaft 808 to rotate through the transmission track 509; and the track tray 708 is used to support the transmission track 509, so that the upper surface of the transmission track 509 is in contact with the lower surface of the rotating roller.

[0069] In the embodiment, the transmission track 509 is located directly below the first rotating roller 809 and the second rotating roller 810, and the upper surface of the transmission track 509 is in contact with the lower surface of the first rotating roller 809 and the second rotating roller 810. The track tray 708 is located directly below the transmission track 509, and the upper surface of the track tray 708 is in contact with the lower surface of the transmission track 509. The transmission track 509 is tensioned by the first tension roller shaft 801 and the second tension roller shaft 808. The rotating roller is embedded in the bearing terminal 711 at both ends, and the bearing terminals at both ends are fixed in turn above the two parallel transmission chains through the roller mounting seat 702. In the embodiment, the upper surface of the track needs to be in contact with the lower surface of the rotating roller. When the transmission track 509 moves, the rotating roller is driven to rotate through friction, the track tray 708 can support the transmission track 509, and the upper surface of the transmission track 509 is ensured to be in contact with the lower surface of the rotating roller.

[0070] In the embodiment, the control mechanism is used to detect whether the material to be detected reaches the detection station, and controls the material conveying mechanism and the material rotating mechanism to switch the working state;

[0071] The control mechanism comprises an embedded controller and a set of light barrier photoelectric switches;

[0072] The control mechanism further comprises a first step driver and a second step driver, and the set of light barrier photoelectric switches comprises a first light barrier photoelectric switch 507 and a second light barrier photoelectric switch 510; the first step motor 704 is connected to the embedded controller through the first step driver, the second step motor 803 is connected to the embedded controller through the second step driver, and the first light barrier photoelectric switch 507 and the second light barrier photoelectric switch 510 are connected to the embedded controller.

[0073] In the embodiment, the embedded controller is an Arduino single-chip microcomputer, which communicates with the detection computing device based on semantic segmentation of the detection system through serial port technology. The first and second step motor drivers control the first and second step motors. The control system is used to control the coordinated movement of the material conveying mechanism, the detection station, the image acquisition mechanism, and the detection system, and communicates using serial port technology. The first and second pair of light barriers 507 and 510 are installed on both sides of the detection station, and the embedded controller acquires the signals emitted by the pair of light barriers to detect whether the material to be detected has reached the detection position.

[0074] In the embodiment, the image acquisition and detection mechanism is used to acquire an image of the material to be detected at an interval of 120° when the material to be detected rotates one revolution, input the acquired three images of the material to be detected into the corn image semantic segmentation network model for online detection of the degree of corn mildew, and calculate the classification of the material to be detected.

[0075] The image acquisition and detection mechanism includes a detection computing device based on semantic segmentation, a camera support 511, and an industrial camera 501.

[0076] As shown in Figure 10 and Figure 11 , the conveying drive mechanism is arranged on the top of the rack, and the two side walls of the rack are parallelly arranged with two packaging side plates 504. A rotating drive mechanism is fixed between the two packaging side plates 504, which drives the movement of the transmission track 509. The upper surface of the transmission track 509 is in contact with the lower surface of the conveying drive mechanism. The camera support 511 is arranged along the center line of the upper surface of the transmission track 509 which is orthogonally projected on the top of the rack, and the center line is perpendicular to the moving direction of the material to be detected. The industrial camera 501 is arranged at the center of the top of the camera support 511. A pair of light barriers is arranged at the two intersection points of the center line and the top of the rack. The pair of light barriers and the industrial camera 501 form a detection plane, which is perpendicular to the upper surface of the conveying drive mechanism. The first and second step motors 704 and 803 and the pair of light barriers are connected to the embedded controller. The detection computing device based on semantic segmentation is connected to the embedded controller and the industrial camera 501.

[0077] In the embodiment, the camera support 511 and the industrial camera 501 form an image acquisition mechanism. The camera support 511 fixes the industrial camera 501 above the detection station, so that the shooting range of the industrial camera 501 can cover the entire detection station. As shown in Figure 9As shown, in the specific implementation process, the industrial camera 501 is fixed to the camera support 511 through the camera fixing base at the middle position of the top middle beam, and the industrial camera 501 is fixed to the detection station through the support. The industrial camera 501 is also located above the middle position of the straight line connected by the first pair of light barriers 507 and the second pair of light barriers 510. The detection plane formed by the first pair of light barriers 507, the second pair of light barriers 510 and the industrial camera 501 is perpendicular to the horizontal direction of the conveying mechanism.

[0078] In the embodiment, the to-be-tested material is lifted by the first rotating roller 809 and the second rotating roller 810 to realize horizontal movement; when the to-be-tested material passes through the detection plane, the first pair of light barriers 507 and the second pair of light barriers 510 are triggered, and the space between the first rotating roller 809 and the second rotating roller 810 is defined as the detection station. Figure 8 As shown, the first rotating roller 809 and the second rotating roller 810 lift the to-be-tested material on the detection station, the upper surface of the transmission track is in contact with the lower surface of the rotating roller, and the corn rotates at a constant speed along the axial direction under the action of the friction force among the transmission track, the rotating roller and the corn. In the embodiment, the third synchronous pulley 805 is coaxially connected with the first tension roller shaft 801, and the first tension roller shaft 801 is coaxially fixed with the third synchronous pulley 805 through the protruding shaft.

[0079] In the embodiment, the control mechanism controls the state switching of the material conveying mechanism and the material rotating mechanism. After the to-be-tested material is conveyed to the detection station by the material conveying mechanism, the to-be-tested material is controlled to rotate 360° along the axial direction by the material rotating mechanism. During the rotation, the image acquisition and detection mechanism shoots a to-be-tested material surface image every 120°, and a total of 3 to-be-tested material surface images are shot during the rotation of 360°. The to-be-tested material surface images are input into the corn image semantic segmentation network model for processing, and the 3 to-be-tested material surface images are merged into a merged picture with a size of 512x512, a unit of pixels and 3 color channels. The merged picture is segmented to obtain a detection result image; the detection result image represents the moldy area, the kernel missing area, the healthy area and the irrelevant background area by different colors; the image acquisition and detection mechanism calculates the corn kernel missing degree Q and the corn mold degree M based on the detection result image, calculates the grading parameter K based on the corn kernel missing degree Q and the corn mold degree M, and calculates the grading of the to-be-tested material based on the grading parameter K. The control mechanism controls the state switching of the material conveying mechanism and the material rotating mechanism again, stops the rotation of the to-be-tested material and moves the to-be-tested material out of the detection station until the next to-be-tested material triggers the first pair of light barriers 507 and the second pair of light barriers 510 again. The cycle is repeated until all to-be-tested materials are detected.

[0080] In the embodiment, the detection computing device based on semantic segmentation is a computer, the detection computing device based on semantic segmentation is a server, the embedded controller is a client, and information interaction between terminals is realized through a serial communication protocol. Specifically, the detection computing device is connected with the embedded controller to control the material conveying mechanism and the detection station, the detection computing device is connected with the image acquisition mechanism to acquire picture data of the material to be detected, and the detection computing device is connected with the detection result output device to output the detection result.

[0081] The image acquisition and detection mechanism calculates and saves the classification of the material to be detected.

[0082] The classification of the material to be detected is:

[0083] When the classification parameter K satisfies 0≤K<0.1, the classification of the material to be detected is defined as level I.

[0084] When the classification parameter K satisfies 0.1≤K<0.3, the classification of the material to be detected is defined as level II.

[0085] When the classification parameter K satisfies 0.3≤K<0.5, the classification of the material to be detected is defined as level III.

[0086] When the classification parameter K satisfies 0.5≤K≤1.0, the classification of the material to be detected is defined as level IV.

[0087] The classification parameter K=Q+M / 2.

[0088] In the formula, Q is the degree of kernel missing, M is the degree of kernel mildew, and Q and M are percentages.

[0089] The corn image semantic segmentation network model comprises a feature extraction part and an up-sampling part, the feature extraction part comprises two convolution layers and one maximum pooling layer, and the up-sampling part comprises one feature fusion layer and two convolution layers.

[0090] The convolution layer uses a 3x3 convolution kernel and does not include a padding operation, and is used to extract local features of an image.

[0091] The maximum pooling layer uses a 2x2 window with a step of 2, so that the feature map size is halved, but the most important feature information is retained.

[0092] The feature fusion layer fuses the feature layer obtained by convolution and the feature layer obtained by up-sampling, and gradually restores the spatial dimension and detail information of the image.

[0093] A CBAM attention mechanism is added between the convolution layer and the feature fusion layer.

[0094] The input of the corn image semantic segmentation network model is a surface image of the material to be detected.

[0095] The output of the corn image semantic segmentation network model is a detection result image obtained by segmenting the input to-be-measured material surface image; the detection result image uses different colors to represent the moldy area, the kernel-lacking area, the healthy area, and the irrelevant background area;

[0096] The to-be-measured material surface image is obtained by splicing three to-be-measured material images collected at intervals of 120°;

[0097] The size of the to-be-measured material surface image is 512x512, the unit is pixel, and the color channel is 3 channels;

[0098] The comprehensive data set used by the corn image semantic segmentation network model includes an original data set and an enhanced data set; the original data set includes a plurality of to-be-measured material surface images; the comprehensive data set is an enhanced image obtained by performing enhancement processing on the to-be-measured material surface images in the original data set; the enhancement processing includes randomly performing a noise increasing operation and a flipping operation on the original to-be-measured material surface image; the noise increasing operation is to randomly increase salt and pepper noise; the flipping operation is to randomly rotate by 45° or 90°;

[0099] The number of pictures in the comprehensive data set is greater than 500, the comprehensive data set is divided into a training set and a test set, and the ratio of the training set to the test set is 9:1;

[0100] The corn image semantic segmentation network model uses the training set for feature extraction; an Adam optimizer is selected, the initial learning rate is defined to be less than or equal to 0.0001, the learning decay rate is 0.5 times the original, and the number of iterations is set to 200 times.

[0101] In an optional embodiment, one end of the material conveying mechanism is provided with a discharging mechanism for realizing vertical falling of a single to-be-measured material through a bottom end opening of a discharging hopper 502 to between adjacent two rotating rollers of the material conveying mechanism, and ensuring that the next to-be-measured material cannot fall when the material conveying mechanism stops moving; the other end of the material conveying mechanism is provided with a discharging buffer tank 508 for storing the to-be-measured material after detection;

[0102] The discharging mechanism includes a discharging hopper 502 and a hopper support 503, the width of the hopper support 503 is greater than the width of the material conveying mechanism; the bottom of the discharging hopper 502 vertically penetrates the opening on the upper surface of the hopper support 503, the discharging hopper 502 is an inverted trapezoidal hopper, the bottom of the discharging hopper 502 is a rectangular opening, and the distance between the bottom opening of the discharging hopper 502 and the rotating roller is 50 mm.

[0103] In this optional embodiment, as Figure 6As shown, the corn discharging mechanism is composed of a funnel support and an inverted trapezoidal discharging funnel. The funnel support is used to support the funnel, so that it is suspended above the material conveying mechanism. The inverted trapezoidal funnel is used to rectify the corn. The bottom of the funnel is a rectangle with a size of 60mm*250mm, and the distance from the bottom of the funnel to the material conveying mechanism below is 50mm. When the conveying device does not move, one corn falls down, and the other corn is pressed and cannot fall down.

[0104] The discharging mechanism is vertically fixed to one end of the material conveying mechanism. A single material to be tested falls vertically through the bottom opening of the discharging funnel 502 to the adjacent two rotating rollers 505 of the material conveying mechanism. The bottom opening of the discharging funnel 502 is arranged to allow only a single corn to fall down. The distance between the bottom opening position and the two rollers of the conveying device is fixed. After the first corn falls between the two rollers, if the conveying device does not move, the next corn cannot fall down. The next corn can only fall between the two rollers after the movement of the previous corn.

[0105] In this alternative embodiment, the material conveying mechanism and the detection station are arranged in the packaging side plate. The discharging mechanism is located directly above the front end of the material conveying mechanism. The image acquisition mechanism is located directly above the detection station. The discharging buffer tank 508 is located at the rear end of the material conveying mechanism.

[0106] In an alternative embodiment, the image acquisition mechanism further comprises two light supplement lamp strips. The two light supplement lamp strips are fixed to the lower sides of the two side beams at the top of the support through light supplement lamp strip fixing pieces. The light supplement lamp strips 512 are fixed to the lower surfaces of the two side beams above the camera support 511 through the first light supplement lamp strip fixing piece 901 and the second light supplement lamp strip fixing piece 907. The light supplement ensures the clarity of the captured images.

[0107] In an alternative embodiment, the image acquisition and detection mechanism is also provided with a scanning gun. The scanning gun is connected to the detection computing device based on semantic segmentation, which realizes the scanning of the batch identification code of the material to be tested, and is used to confirm the type and batch of the material to be tested.

[0108] In an alternative embodiment, the image acquisition and detection mechanism is also provided with a display device. The display device is connected to the detection computing device based on semantic segmentation, which realizes the real-time output of the detection results. The detection results include: the batch, quantity, corn mildew degree, corn kernel deficiency degree and classification of the material to be tested.

[0109] In this alternative embodiment, the display device is an external display, which facilitates real-time observation of the detection results.

[0110] The corn mildew degree online detection system based on semantic segmentation aims to provide a platform architecture for online detection methods, combines mechatronic automatic control, realizes automatic unloading, conveying and detection of corn, and develops a detection system software that can realize corn mildew detection visualization, calculate and display the mildew degree, etc.

[0111] In one specific embodiment, after the corn mildew degree online detection system based on semantic segmentation is powered on, the batch identification code of corn is scanned and recognized, the corn is put into the unloading hopper 502, and the corn to be detected falls between the adjacent two rotating rollers of the material conveying mechanism in turn. The control system controls the first stepper motor 704 to rotate, the first stepper motor 704 drives a group of transmission sprockets 706 to rotate through the first synchronous pulley 705, and the first transmission chain 701 drives the rotating rollers to move at a uniform speed in turn. After the corn to be detected falls between the first rotating roller 809 and the second rotating roller 810 from the corn unloading hopper 502, it reaches the detection station under the action of the transmission sprocket 706, triggers the first pair of reflective photoelectric switches 507 and the second pair of reflective photoelectric switches 510, and the control system controls the first stepper motor 704 to stop moving. The second stepper motor 803 starts to rotate, and the track tensioning roller shaft 801 moves under the action of the second synchronous pulley 804. The tensioning transmission track 509 rotates under the action of friction, and the first rotating roller 809 and the second rotating roller 810 rotate along the axis under the friction of the upper surface of the transmission track 509 and the lower surface of the first rotating roller 809 and the second rotating roller 810. The corn to be detected also starts to rotate 360° along the axis at the detection station under the action of friction. In this process, a corn surface image is taken every 120°, and a total of 3 corn surface images are taken after rotating 360°. While the control system controls as described above, it interacts with the detection system in real time through the serial port. The detection system controls the industrial camera 501 to start collecting corn surface images and inputs the detection model for detection. After the detection is completed, the control system controls the second stepper motor 803 to stop rotating, and the first stepper motor 704 continues to rotate until the next pair of reflective photoelectric switches 507, 510 are triggered.

[0112] The detection system controls the industrial camera 501 to collect a surface image every 120° of rotation of the corn during the rotation of the corn, three surface images of the corn are collected during the entire rotation of the corn for cropping and splicing, as a complete surface image of the corn to be detected, and input to the detection model for detection, after the detection process is completed, the complete surface image and the complete result surface image after detection are displayed on the detection software and saved, wherein the complete result surface image after detection, the dark yellow area represents the healthy corn area, the green area represents the mildewed corn area, and the light yellow area represents the kernel-deficient corn area. The detection and grading results are saved, and a statistical result table file is generated under the root directory, the statistical result table file content includes detected corn batch information, total pixel area of the corn, kernel-deficient degree of the corn, mildew degree of the corn, and corn grading level.

[0113] The corn grading standard is determined by two indexes of the kernel-deficient degree Q and the mildew degree M, and a grading parameter K=(Q+M) / 2, wherein Q and M are percentages. When the value of K is between 0 and 0.1, it is defined as grade I; when the value of K is between 0.1 and 0.3, it is defined as grade II; when the value of K is between 0.3 and 0.5, it is defined as grade III; and when the value of K is between 0.5 and 1, it is defined as grade IV.

[0114] As shown in Figure 5 Another embodiment of the present application discloses a detection method of the online detection system for the mildew degree of corn based on semantic segmentation according to the present application, which comprises the following steps:

[0115] The material to be detected falls between the adjacent first rotating roller 809 and second rotating roller 810 on the material conveying mechanism in sequence;

[0116] The control mechanism controls the rotation of the first stepping motor 704, the first stepping motor 704 drives a group of transmission sprockets 706 to rotate through the first synchronous pulley 705, and the first transmission chain 701 drives the rotating rollers to move uniformly in sequence; the material to be detected is lifted by the adjacent first rotating roller 809 and second rotating roller 810 to realize horizontal movement;

[0117] When the to-be-tested material passes through the detection plane, the first pair of photoelectric switches 507 and the second pair of photoelectric switches 510 are triggered, the control mechanism judges that the to-be-tested material reaches the detection station, and controls the first stepper motor 704 to stop rotating and the second stepper motor 803 to start rotating. Under the action of the second synchronous pulley 804, the track tension roller shaft 801 is driven to move, and the transmission track 509 is rotated under the friction force. Under the friction force between the upper surface of the transmission track 509 and the lower surfaces of the first rotating supporting roller 809 and the second rotating supporting roller 810, the first rotating supporting roller 809 and the second rotating supporting roller 810 rotate along the axial direction. Under the friction force, the to-be-tested material starts to rotate 360° along the axial direction at the detection station. In this process, the image acquisition and detection mechanism takes a picture of the surface of the to-be-tested material every 120°, and a total of 3 pictures of the surface of the to-be-tested material are taken in 360° rotation;

[0118] The surface image of the to-be-tested material is input into the corn image semantic segmentation network model for processing, and 3 surface images of the to-be-tested material are merged into one merged picture with a size of 512*512, a unit of pixels, and 3 color channels. The merged picture is segmented to obtain a detection result image. The detection result image represents the moldy area, the kernel-lacking area, the healthy area, and the irrelevant background area by different colors. Based on the detection result image, the image acquisition and detection mechanism calculates the kernel-lacking degree Q of the corn and the moldy degree M of the corn, calculates the grading parameter K based on the kernel-lacking degree Q of the corn and the moldy degree M of the corn, and calculates the classification of the to-be-tested material based on the grading parameter K.

[0119] The image acquisition and detection mechanism sends a signal to the control mechanism, the control mechanism controls the second stepper motor 803 to stop rotating, and the first stepper motor 704 continues to rotate to move the to-be-tested material out of the detection station until the next to-be-tested material triggers the first pair of photoelectric switches 507 and the second pair of photoelectric switches 510 again. This cycle continues until all to-be-tested materials are detected.

[0120] In this embodiment, the semantic segmentation-based detection calculation device controls the first stepper driver through the embedded controller to drive the first stepper motor 704 to rotate. The first transmission chain 701, the plurality of rotating supporting rollers, and the second transmission chain form a conveying plane. Under the drive of the first stepper motor 704, a single to-be-tested material moves along the horizontal direction of the conveying plane under the lifting of adjacent two rotating supporting rollers.

[0121] The first pair of photoelectric switches 507 and the second pair of photoelectric switches 510 are fixed on both sides of the detection station through the photoelectric switch mounting base 506. The first pair of photoelectric switches 507, the second pair of photoelectric switches 510, and the industrial camera 501 form a detection plane, which is perpendicular to the conveying plane.

[0122] When the to-be-tested material passes through the detection plane, the first pair of light beam photoelectric switches 507 and the second pair of light beam photoelectric switches 510 will start to transmit a start detection signal to the detection computing device through the embedded controller, and the embedded controller will stop the first stepper motor 704 through the first stepper driver to achieve the parking of the single to-be-tested material within the shooting range of the industrial camera 501;

[0123] When the detection computing device receives the start detection signal, the detection computing device controls the second stepper driver through the embedded controller to drive the second stepper motor 803 connected to the second stepper driver to move the transmission track 509 in the horizontal direction to drive the rotating roller to rotate in the axial direction, and the single to-be-tested material is driven to rotate in the axial direction through the rotating roller rotating in the axial direction, and the industrial camera 501 is controlled by the detection computing device to shoot the to-be-tested material to obtain a plurality of image data; the detection computing device based on semantic segmentation detects the plurality of image data to obtain a detection result and sends an end detection signal; and the detection result is realized by the result output device;

[0124] In a specific embodiment, the detection method of the corn mold degree online detection system based on semantic segmentation is implemented as follows:

[0125] The batch identification code of the to-be-tested material is scanned to confirm the type and batch of the to-be-tested material;

[0126] After the to-be-tested material is automatically discharged by the discharging mechanism, the single to-be-tested material is transported to the detection station by the material conveying mechanism;

[0127] After reaching the detection station, the industrial camera 501 starts to collect an image every 120° of rotation of the to-be-tested material rotating at a constant speed in the axial direction, a total of three to-be-tested material surface images, and inputs the collected three to-be-tested material surface images into the image collection and detection mechanism;

[0128] The image collection and detection mechanism splices the three to-be-tested material surface images into one image after cutting along the axial direction, and inputs the to-be-tested material surface image into the corn image semantic segmentation network model as a to-be-tested material surface image for regional detection by the corn image semantic segmentation network model;

[0129] After the detection is completed, the complete surface image of the to-be-tested material, the processed complete surface image, and the detection result are displayed on the software interface;

[0130] The image collection and detection mechanism saves and counts the detection result and the grading result of the to-be-tested material according to the grading index;

[0131] The material conveying mechanism transports the detected to-be-tested material out of the detection station.

[0132] The detection result image represents the mildew area, the kernel missing area, the healthy area and the irrelevant background area by different colors; the image acquisition and detection mechanism calculates the corn kernel missing degree Q and the corn mildew degree M based on the detection result image, calculates the grading parameter K based on the corn kernel missing degree Q and the corn mildew degree M, and calculates the material to be graded based on the grading parameter K;

[0133] In the embodiment, the corn kernel missing degree Q and the corn mildew degree M are obtained by:

[0134] The number of corn kernel missing pixels is defined as q; the number of corn mildew pixels is defined as m; the number of corn normal pixels is defined as n; the total number of effective pixels t=q+m+n;

[0135] The color value of each pixel point of the detection result image is traversed; if the color value is equal to the pre-set corn kernel missing color value, the number of corn kernel missing pixels q is increased by 1; if the color value is equal to the pre-set corn mildew color value, the number of corn mildew pixels m is increased by 1; if the color value is equal to the pre-set corn normal color value, the number of corn normal pixels n is increased by 1;

[0136] The corn kernel missing degree Q=q÷t is calculated;

[0137] The corn mildew degree M=m÷t is calculated;

[0138] In the embodiment, as shown in the software operation interface, Figure 4 is designed by QtDesigner and is programmed by python language to realize corresponding functions, is designed to run in multiple threads to prevent infinite loop program from blocking threads, and can realize the functions of statistics, display and saving of the batch, quantity, mildew degree, kernel missing degree and corn grading of the detected corn.

[0139] The material to be graded is:

[0140] When the grading parameter K satisfies 0≤K<0.1, the material to be graded is defined as grade I;

[0141] When the grading parameter K satisfies 0.1≤K<0.3, the material to be graded is defined as grade II;

[0142] When the grading parameter K satisfies 0.3≤K<0.5, the material to be graded is defined as grade III;

[0143] When the grading parameter K satisfies 0.5≤K≤1.0, the material to be graded is defined as grade IV;

[0144] The grading parameter K=Q+M / 2;

[0145] In the formula, Q is the corn kernel missing degree, M is the corn mildew degree, and Q and M are both percentages.

[0146] The corn image semantic segmentation network model comprises a feature extraction part and an up-sampling part, the feature extraction part comprises two convolution layers and one maximum pooling layer, and the up-sampling part comprises one feature fusion layer and two convolution layers;

[0147] The convolution layer uses a 3x3 convolution kernel and does not comprise a padding operation, and is used for extracting local features of an image;

[0148] The maximum pooling layer uses a 2x2 window with a step of 2, so that the feature map size is halved, but the most important feature information is retained;

[0149] The feature fusion layer fuses the feature layer obtained through convolution and the feature layer obtained through up-sampling, and gradually restores the spatial dimension and detail information of the image;

[0150] A CBAM attention mechanism is added between the convolution layer and the feature fusion layer;

[0151] The input of the corn image semantic segmentation network model is a surface image of a material to be detected;

[0152] The output of the corn image semantic segmentation network model is a detection result image obtained by segmenting the input surface image of the material to be detected; the detection result image uses different colors to represent a moldy area, a kernel-lacking area, a healthy area and an irrelevant background area;

[0153] The surface image of the material to be detected is obtained by splicing three surface images of the material to be detected collected at intervals of 120°;

[0154] The size of the surface image of the material to be detected is 512x512, the unit is pixel, and the color channel is 3 channels;

[0155] The comprehensive data set used by the corn image semantic segmentation network model comprises an original data set and an enhanced data set; the original data set comprises a plurality of surface images of the material to be detected; the comprehensive data set is an enhanced image obtained by performing enhancement processing on the surface images of the material to be detected in the original data set; the enhancement processing comprises randomly performing a noise increasing operation and a flipping operation on the original surface image of the material to be detected; the noise increasing operation is to randomly increase salt and pepper noise; the flipping operation is to randomly rotate by 45° or 90°;

[0156] The number of pictures in the comprehensive data set is greater than 500, the comprehensive data set is divided into a training set and a test set, and the ratio of the training set to the test set is 9:1;

[0157] The corn image semantic segmentation network model uses a training set for feature extraction; an Adam optimizer is selected, the initial learning rate is defined to be less than or equal to 0.0001, the learning decay rate is 0.5 times the original, and the iteration number is set to 200 times.

[0158] In this embodiment, the specific process of constructing the corn image semantic segmentation network model is as shown in Figure 3

[0159] The corn image semantic segmentation network model includes a feature extraction part and an up-sampling part, the feature extraction part contains 2 convolution layers and 1 max pooling layer, and the up-sampling part contains 1 feature fusion layer and 2 convolution layers. The corn image semantic segmentation network model, the first half is feature extraction, and the second half is up-sampling. The feature extraction part is composed of convolution operation and down-sampling operation, and the convolution structure used is uniformly 3x3 convolution kernel. The up-sampling part is used to restore the original resolution of the feature map, and is commonly implemented by transposed convolution and interpolation. In the process of model training, high gradient descent is used, so that the subsequent learning rate of the trained model is in a self-adjusting process. The corn image semantic segmentation network model has the characteristics of simple structure and low construction difficulty, and good results can be achieved without a large amount of data set training.

[0160] In this embodiment, the trained network model is deployed to a corn mold degree online detection system based on semantic segmentation, and is connected to the camera hardware and interacts with the control system through serial communication, and cooperates with the mechanical structure of the device to realize the detection of the mold degree of corn.

[0161] In this embodiment, as shown in Figure 12 The corn image semantic segmentation network model is mainly divided into left and right parts, of which the left part is the feature extraction part and the right part is the up-sampling part.

[0162] Convolution layer (Conv2d): The convolution layer is used to extract local features of the image. A 3x3 convolution kernel is used, and no padding operation (padding=0) is included, resulting in a reduction in the size of the feature map after each convolution, while the number of channels increases, keeping the spatial dimension unchanged.

[0163] Max pooling layer (Max Pool): The max pooling layer uses a 2x2 window with a step of 2, which reduces the feature map size by half, but retains the most important feature information.

[0164] Feature fusion layer (Concatenate): The feature fusion layer fuses the feature layers obtained by convolution and the feature layers obtained by up-sampling, gradually restoring the spatial dimension and detail information of the image. The feature fusion layer is the key to connecting the two parts of the model, allowing different levels of feature information to interact and fuse, thereby fully utilizing the rich information in the image.​

[0165] In this embodiment, as Figure 13 As shown, a CBAM attention mechanism is added between the convolutional layer and the feature fusion layer. The CBAM attention mechanism takes into account both spatial and channel aspects to resample the input data and strengthen specific objects. In spatial attention, the output is obtained by multiplying the point-state matrix between the differentiable weight mask and the convolutional feature map. In channel attention, the feature channels are weighted and calculated.

[0166] In this embodiment, adding a CBAM attention mechanism between the convolutional layer and the feature fusion layer has the following advantages:

[0167] 1. Enhanced feature representation: Through a two-dimensional attention mechanism of channel and space, the input feature layer is automatically selected, and features useful for the task are enhanced while unimportant background features are suppressed.

[0168] 2. Improve model performance: Make the model more focused on key features and reduce noise interference, thereby improving the model's accuracy and robustness.

[0169] 3. Optimize feature fusion: Enhance the feature map before feature fusion to make the fused features more effective and promote the interaction between features at different levels.

[0170] In an optional embodiment, without using a semantic segmentation-based online corn mold detection system to collect corn images, a semantic segmentation-based detection computing device can also directly analyze and detect archived corn mold images.

[0171] In this embodiment, the image of the material to be tested is a surface image of corn. The corn surface image is acquired by placing the corn in the detection station and using the transmission belt and rotating rollers at the detection station to drive the corn to rotate uniformly along the axial direction. The camera acquires the original image of the corn, and the number of corn images of various types is greater than 500. Each image is 512×512 pixels in size and has 3 color channels.

[0172] The original images are processed by image processing methods, including batch random addition of noise and flipping operations to the original dataset to obtain an enhanced dataset, which is then merged with the original dataset to form a comprehensive dataset. Enhancement methods include random rotations of 45° and 90°. The random noise addition operation involves randomly adding salt-and-pepper noise.

[0173] Salt and pepper noise, also known as impulse noise, is a black and white alternating bright and dark spot noise generated by image sensors, transmission channels, decoding processes, etc., through random changes in some pixel values.

[0174] The comprehensive data set is divided into a training set and a test set in proportion, and the proportion of the training set and the test set is 9:1.

[0175] In the data set making process, the data set is data enhanced to improve the accuracy of identifying the moldy corn area; in the device running process, one camera is used to collect three images in the 360° rotation process of the corn, and one complete surface image of the corn is obtained after cutting and splicing every three images. The above implementation mode can detect the moldy degree of each corn in all directions and completely, reduce the misjudgment rate, and improve the result reliability.

[0176] The training set is used as the input of the detection model for feature extraction, and the network is trained for a total of K rounds, K=200, the input image size is 512x512, 3 channels of RGB color image, Adam optimizer is selected, the initial learning rate is defined to be not greater than 0.0001, the learning decay rate is 0.5 times of the original, and the iteration number is set to 200 times.

[0177] The network weight file corresponding to the model with the best average precision value effect in the training stage is used as the final model to obtain the corn image semantic segmentation network model.

[0178] In this embodiment, the interval of 120 degrees is because the corn rotates 360 degrees, and the three pictures collected at an interval of 120 degrees have a smaller deviation from the overall surface of the corn, and the processing process is fast, which can meet the real-time detection demand of the device; the initial learning rate is defined to be less than or equal to 0.0001, and the learning decay rate is 0.5 times of the original, in order to make the loss curve of the model converge faster during training and speed up the training process; the iteration number is set to 200 times, because when the training reaches the 200th time, the loss rate of the model tends to be stable, so the training is stopped.

[0179] In this embodiment, the corn image semantic segmentation network model realizes pixel-level segmentation for moldy, healthy and kernel-deficient areas of the input corn image, and can obtain higher segmentation accuracy; the output image after detection is clear and understandable, the dark green color represents the moldy area, the light yellow color represents the kernel-deficient area, the dark yellow color represents the healthy area, and the black area represents the irrelevant background. Compared with the prior art which only segments the defect area, the corn image semantic segmentation network model processes the input image, removes the redundant background information, makes the detection result image clearer and more understandable, and outputs and saves the kernel-deficient degree of the corn, so that the corn image semantic segmentation network model detects the corn more accurately and the classification index and basis are more comprehensive.

[0180] The detection method of the corn mildew degree online detection system based on semantic segmentation collects images of various healthy, lack of particle and mildew corns and labels various regions, expands the data set through a data enhancement method, divides the data set into a training set and a test set according to a proportion, inputs a training model to obtain optimal weights, constructs a corn mildew degree detection model and builds a detection system, and a control system controls the cooperative movement of a device and the detection system through a serial communication protocol to realize information interaction between terminals. The detection system realizes automatic corn material unloading and detection, code scanning and identification of corn batches, display of detection results and storage, grading of corns and local picture detection, solves problems such as large labor intensity, long time consumption and low efficiency, and has good social and economic benefits.

Claims

1. An online detection system for the degree of mold growth in corn based on semantic segmentation, characterized in that, include: Material conveying mechanism, material rotation mechanism, control mechanism, and image acquisition and detection mechanism; The material conveying mechanism includes: a frame and a conveying drive mechanism, wherein the conveying drive mechanism includes a first stepper motor (704); the material rotation mechanism includes: a transmission track (509) and a rotation drive mechanism, wherein the rotation drive mechanism includes a second stepper motor (803); the control mechanism includes: an embedded controller and a through-beam photoelectric switch group; the image acquisition and detection mechanism includes: a semantic segmentation-based detection computing device, a camera bracket (511), and an industrial camera (501). The conveying drive mechanism is located on the top of the frame, and two encapsulation side plates (504) are arranged parallel to each other on the two side walls of the frame. A rotary drive mechanism is fixed between the two encapsulation side plates (504), and the rotary drive mechanism drives the transmission track (509) to move. The upper surface of the transmission track (509) is in contact with the lower surface of the conveying drive mechanism. A camera bracket (511) is set along the center line of the orthographic projection of the upper surface of the transmission track (509) on the top of the frame. The center line is perpendicular to the direction of movement of the material to be measured. An industrial camera (501) is set at the top center of the camera bracket (511). A through-beam photoelectric switch group is set at the two intersections of the center line and the top of the frame. The through-beam photoelectric switch group and the industrial camera (501) form a detection plane. The detection plane is perpendicular to the upper surface of the conveying drive mechanism. The first stepper motor (704), the second stepper motor (803), and the through-beam photoelectric switch group are respectively connected to the embedded controller. The semantic segmentation-based detection computing device is respectively connected to the embedded controller and the industrial camera (501). The material conveying mechanism is used to transport the material to be tested; the material rotating mechanism is used to control the material to be tested to rotate one revolution; the control mechanism is used to detect whether the material to be tested has reached the detection station and to control the material conveying mechanism and the material rotating mechanism to switch working states; the image acquisition and detection mechanism is used to acquire an image of the material to be tested at 120° intervals when the material to be tested rotates one revolution, and input the three images of the material to be tested into the corn image semantic segmentation network model to perform online detection of the degree of corn mold and calculate the grade of the material to be tested. The corn image semantic segmentation network model includes a feature extraction part and an upsampling part. The feature extraction part contains two convolutional layers and one max pooling layer, and the upsampling part contains one feature fusion layer and two convolutional layers. The convolutional layer uses a 3x3 convolutional kernel and does not include padding operations; it is used to extract local features of the image. The max pooling layer uses a 2x2 window with a stride of 2, which halves the feature map size but retains the most important feature information. The feature fusion layer fuses the feature layers obtained by convolution with the feature layers obtained by upsampling, gradually restoring the spatial dimension and detail information of the image. Add a CBAM attention mechanism between the convolutional layer and the feature fusion layer; The input to the corn image semantic segmentation network model is the surface image of the material to be tested; The output of the corn image semantic segmentation network model is to segment the surface image of the input material to be tested to obtain the detection result image; the detection result image uses different colors to represent moldy areas, missing kernel areas, healthy areas and irrelevant background areas; The surface image of the material to be tested is obtained by stitching together three images of the material to be tested acquired at 120° intervals. The image acquisition and detection mechanism calculates the degree of corn kernel loss Q and the degree of corn mold M based on the detection result image, calculates the grading parameter K based on the degree of corn kernel loss Q and the degree of corn mold M, and calculates the grading of the material to be tested based on the grading parameter K.

2. The online detection system for the degree of mold growth in corn based on semantic segmentation according to claim 1, characterized in that, The transmission drive mechanism further includes: a first synchronous pulley (705), a first synchronous belt (703), a first transmission sprocket (706), a second transmission sprocket (710), a first transmission chain (701), a third transmission sprocket, a fourth transmission sprocket, a second transmission chain, and a chain tray (707); a first stepper motor (704) drives the first synchronous pulley (705) to rotate. The first synchronous belt (703) drives the fourth synchronous pulley (601), which in turn drives the first transmission sprocket (706) to rotate. The first transmission sprocket (706) drives the second transmission sprocket (710) to rotate via the first transmission chain (701). The first transmission sprocket (706) and the third transmission sprocket are coaxial, and the third transmission sprocket drives the fourth transmission sprocket to rotate via the second transmission chain. Several roller mounting seats (702) are evenly spaced on the outer surfaces of the parallel first transmission chain (701) and the second transmission chain. Bearing terminals (711) are embedded at both ends of each rotating roller, and the bearing terminals (711) are fixedly connected to the roller mounting seats (702) at the opposite positions. The distance between two adjacent rotating rollers is less than the width of a single material to be tested. The chain tray (707) is used to support the first transmission chain (701) and the second transmission chain.

3. The online detection system for the degree of mold growth in corn based on semantic segmentation according to claim 1, characterized in that, The rotary drive mechanism further includes: a second synchronous pulley (804), a second synchronous belt (802), a third synchronous pulley (805), a first tension roller shaft (801), a second tension roller shaft (808), and a track tray (708); a second stepper motor (803) drives the second synchronous pulley (804) to rotate; the second synchronous pulley (804) drives the third synchronous pulley (805) to rotate through the second synchronous belt (802); the third synchronous pulley (805) is coupled to the first tension roller shaft (801) and drives the first tension roller shaft (801) to rotate; the first tension roller shaft (801) drives the second tension roller shaft (808) to rotate through the transmission track (509); the track tray 708 is used to support the transmission track (509) so that the upper surface of the transmission track (509) contacts the lower surface of the rotating idler.

4. The online detection system for the degree of mold growth in corn based on semantic segmentation according to claim 1, characterized in that, The control mechanism further includes: a first stepper driver and a second stepper driver. The through-beam photoelectric switch group includes: a first through-beam photoelectric switch (507) and a second through-beam photoelectric switch (510). The first stepper motor (704) is connected to the embedded controller through the first stepper driver, the second stepper motor (803) is connected to the embedded controller through the second stepper driver, and the first through-beam photoelectric switch (507) and the second through-beam photoelectric switch (510) are connected to the embedded controller.

5. The online detection system for the degree of mold growth in corn based on semantic segmentation according to claim 1, characterized in that, One end of the material conveying mechanism is provided with a feeding mechanism, which is used to realize that a single material to be tested falls vertically through the bottom opening of the feeding funnel (502) to the space between two adjacent rotating rollers of the material conveying mechanism, and ensures that when the material conveying mechanism stops moving, the next material to be tested cannot fall; the other end of the material conveying mechanism is provided with a feeding buffer box (508), which is used to store the material to be tested after the test is completed. The feeding mechanism includes a feeding funnel (502) and a funnel support (503). The width of the funnel support (503) is greater than the width of the material conveying mechanism. The bottom of the feeding funnel (502) passes vertically through the opening on the upper surface of the funnel support (503). The feeding funnel (502) is an inverted trapezoidal funnel. The bottom of the feeding funnel (502) has a rectangular opening. The distance between the bottom opening of the feeding funnel (502) and the rotating roller is 50mm.

6. The online detection system for the degree of mold growth in corn based on semantic segmentation according to claim 1, characterized in that, The image acquisition and detection mechanism is also equipped with a scanner, which is connected to a semantic segmentation-based detection computing device to scan the batch identification code of the material to be tested, thereby confirming the type and batch of the material to be tested.

7. The online detection system for the degree of mold growth in corn based on semantic segmentation according to claim 1, characterized in that, The image acquisition and detection mechanism is also equipped with a display device, which is connected to a semantic segmentation-based detection computing device to realize real-time output of detection results. The detection results include: batch and quantity of the detected material, degree of corn mold, degree of corn kernel defects, and grade of the material to be tested.

8. The online detection system for the degree of mold growth in corn based on semantic segmentation according to claim 1, characterized in that, The material to be tested is classified as follows: When the grading parameter K satisfies: 0≤K<0.1, the material to be tested is defined as Grade I; When the grading parameter K satisfies the condition that 0.1 ≤ K < 0.3, the material to be tested is defined as Grade II. When the grading parameter K satisfies the condition that 0.3 ≤ K < 0.5, the material to be tested is defined as Grade III. When the grading parameter K satisfies: 0.5≤K≤1.0, the material to be tested is defined as grade IV; The grading parameter K = (Q + M) / 2; In the formula, Q represents the degree of missing kernels in the corn, M represents the degree of mold in the corn, and both Q and M are percentages.

9. The online detection system for the degree of mold growth in corn based on semantic segmentation according to claim 1, characterized in that, The size of the surface image of the material to be tested is 512×512 pixels, and the color channel is 3 channels. The comprehensive dataset used by the corn image semantic segmentation network model includes: an original dataset and an augmented dataset; the original dataset includes several surface images of the material to be tested; the comprehensive dataset is an augmented image obtained by augmenting the surface images of the material to be tested in the original dataset; the augmentation process includes: randomly adding noise and flipping the original surface images of the material to be tested; the noise addition operation is to randomly add salt and pepper noise; the flipping operation is to randomly rotate 45° or 90°. The comprehensive dataset contains more than 500 images. The comprehensive dataset is divided into a training set and a test set, with a ratio of 9:1 between the training set and the test set. The corn image semantic segmentation network model uses the training set for feature extraction; the Adam optimizer is selected, the initial learning rate is defined to be less than or equal to 0.0001, the learning decay rate is 0.5 times the original, and the number of iterations is set to 200.

10. A detection method for an online corn mold degree detection system based on semantic segmentation according to any one of claims 1-9, characterized in that, Includes the following steps: The material to be tested falls sequentially between the first rotating roller (809) and the second rotating roller (810) on the material conveying mechanism; The control mechanism controls the first stepper motor (704) to rotate. The first stepper motor (704) drives a set of transmission sprockets (706) to rotate through the first synchronous belt pulley (705). The first transmission chain (701) then drives the rotating rollers to move at a constant speed in sequence. The material to be tested is lifted by the adjacent first rotating roller (809) and second rotating roller (810) to achieve horizontal movement. When the material to be tested passes through the detection plane, the first through-beam photoelectric switch (507) and the second through-beam photoelectric switch (510) are triggered. The control mechanism determines that the material to be tested has reached the detection station and controls the first stepper motor (704) to stop rotating and the second stepper motor (803) to start rotating. Under the action of the second synchronous pulley (804), the track tensioning roller shaft (801) is driven to move. The tensioned transmission track (509) rotates under the action of friction. Under the action of friction between the upper surface of the transmission track (509) and the lower surface of the first rotating roller (809) and the second rotating roller (810), the first rotating roller (809) and the second rotating roller (810) will rotate in the same direction along the axial direction. Under the action of friction, the material to be tested starts to rotate 360° along the axial direction at the detection station. During this process, the image acquisition and detection mechanism takes a picture of the surface of the material to be tested every 120°. A total of 3 pictures of the surface of the material to be tested are taken after rotating 360°. The surface image of the material to be tested is input into the corn image semantic segmentation network model for processing. The three surface images of the material to be tested are merged into a single image with a size of 512×512 pixels and 3 color channels. The merged image is then segmented to obtain the detection result image. The detection result image uses different colors to represent moldy areas, missing kernel areas, healthy areas, and irrelevant background areas. The image acquisition and detection mechanism sends a signal to the control mechanism, which controls the second stepper motor (803) to stop rotating, while the first stepper motor (704) continues to rotate, moving the material to be tested out of the detection station until the next material to be tested triggers the first through-beam photoelectric switch (507) and the second through-beam photoelectric switch (510) again. This cycle continues until all materials to be tested have been detected.

Images