A method and system for automatically grading stropharia

Abstract

The present invention discloses an automatic grading method and system for stropharia, the system comprising a feeding system, a transmission system, an image acquisition visual system and a grading system; the feeding system is used to sequentially and evenly deliver the stropharia to be graded to the starting end of the transmission system; the transmission system is used to sequentially transmit the stropharia to be graded to the working range of the image acquisition visual system and the grading system, and move the stropharia that have completed the level classification out of the working range of the image acquisition visual system; the image acquisition visual system is used to extract and analyze the information of the stropharia, and transmit the acquired information to the grading system; the grading system is used to grade the stropharia according to the information conveyed by the image acquisition visual system and the grading method, and blow the stropharia into the collection frame through the trachea. The present invention uses a convolutional neural model and an image algorithm to grade the quality of stropharia, and this method is simple, effective and efficient, and has a very high recognition accuracy rate for stropharia, ensuring the accuracy of automatic grading.

Description

Technical Field

[0001] The invention relates to the fields of electromechanical control and machine vision, and in particular to an automatic grading method and system for Stropharia rugosa. Background Art

[0002] Stropharia, also known as red pine mushroom, wrinkled globe mushroom, wine red Stropharia. This mushroom species is rich in nutrients and has good medicinal and health value. As an emerging edible mushroom variety, it has tender meat, fragrant taste, delicious taste, and is rich in 18 kinds of amino acids, multiple minerals and trace elements, as well as multiple biologically active ingredients, such as polysaccharides, sterols and flavonoids. Compared with common edible mushrooms such as Flammulina velutipes, Pleurotus eryngii, and Oyster mushroom, it has higher nutritional value and is crowned with the reputation of "meat in vegetarian food". It has broad development prospects. At present, the post-harvest grading of Stropharia depends entirely on manual sorting. Due to differences in judgment among different personnel, it is easy to produce mixed packing of Stropharia of different grades, which reduces market competitiveness. Strictly sorting and grading of Stropharia in accordance with standards can avoid quality fluctuations caused by cascading, achieve high quality and high price, and thus improve the commercialization and value of Stropharia.

[0003] In recent years, with the continuous progress and innovation of artificial intelligence, the use of convolutional neural models combined with automated equipment for crop grading has been proven to be effective. The convolutional neural model uses convolution kernels to effectively extract the features of the target object. Through the precise analysis and discrimination of the algorithm, it has shown great potential in the field of fruit and vegetable appearance quality grading detection.

[0004] At present, the research on the grading and detection of mushrooms is still limited to the use of different image processing algorithms combined with automated equipment for grading operations. In terms of algorithms, most scholars at home and abroad use relatively traditional methods for the detection and grading of mushroom agricultural products, and most of them only focus on the size of mushrooms, with little research on the appearance details, resulting in unsatisfactory grading results; in terms of equipment, the grading equipment used for mushroom grading operations is bulky and complex, and often causes varying degrees of mechanical damage to the mushrooms during the grading process, which will seriously affect the appearance quality of the mushrooms, thereby reducing their market competitiveness.

[0005] Based on the above situation, it is necessary to study an automatic grading method and system that can accurately and efficiently identify the giant puffball mushroom and is simple to build and not easy to damage the giant puffball mushroom. Summary of the invention

[0006] In order to solve the above problems, the purpose of the present invention is to provide an automatic grading method and system for Stropharia rugosa, which has the advantages of high efficiency and high accuracy.

[0007] In order to achieve the above object, the present invention provides the following scheme:

[0008] An automatic grading method for Stropharia rugosotis, the method comprising

[0009] Acquire images of Stropharia officinalis on a transmission system;

[0010] Perform image distortion correction on the acquired Stropharia rugosa images;

[0011] After annotating the corrected Stropharia images, a corresponding data set is created, which is then passed into the convolutional neural model to obtain the segmentation mask images of the Stropharia cap and stipe;

[0012] Performing color separation according to the segmentation mask image of the cap and stipe of the Stropharia capifolia to obtain segmentation images of the cap and stipe with different colors;

[0013] Performing image processing on the segmented images of the caps and stipes with different colors to obtain contour images of the separated caps and stipes of the Stropharia capifolia and their feature information, and the minimum circumscribed rectangle of the separated caps and stipes of the Stropharia capifolia and their coordinate information;

[0014] According to the minimum circumscribed rectangle of the separated cap and stipe of the Stropharia capifolia and its coordinate information, whether the center line connecting the minimum circumscribed rectangle of the cap and stipe is vertical is judged to determine whether to rotate the image of the separated cap and stipe of the Stropharia capifolia, and finally obtain the vertical contour image of the separated cap and stipe of the Stropharia capifolia and its characteristic information;

[0015] According to the outline image of the vertical Stropharia cap and stipe separated and its characteristic information and the minimum circumscribed rectangle and its coordinate information, a detection model of the diameter-to-width ratio of the cap (RDHP) and the diameter-to-length ratio of the stipe (RLDS) is finally established, so as to determine the grade of the Stropharia cap according to the grading standard.

[0016] Optionally, color separation is performed according to the segmentation mask image of the cap and stipe of the Stropharia capifolia to obtain segmented images of the cap and stipe with different colors, specifically comprising:

[0017] Processing the cap and stipe of the Stropharia capensis with segmentation masks of different colors according to the convolutional neural model;

[0018] The separation mask images of the cap and the stipe of the Stropharia cap are color separated according to the different color segmentation masks.

[0019] Optionally, performing image processing on the segmented images of the caps and stipes with different colors to obtain the minimum circumscribed rectangle of the separated caps and stipes of the Stropharia capensis and their coordinate information specifically includes:

[0020] Binarization is performed on the segmentation mask images of the caps and stipes with different colors, and all non-0 pixels are set to 1 to obtain binary images of the caps and stipes;

[0021] Extracting contour features of the binary images of the cap and stipe respectively to obtain contour images of the cap and stipe and their characteristic information;

[0022] The minimum rectangular frame is obtained according to the outline image of the cap and the stipe, and the minimum circumscribed rectangle surrounding the outline image of the cap and the stipe of the Stropharia capensis and its coordinate information are obtained.

[0023] Optionally, the step of judging whether the center line connecting the minimum circumscribed rectangle of the cap and the stipe is vertical to determine whether the images of the cap and stipe separated from the Stropharia capifolia are rotated specifically includes:

[0024] Extract the minimum rectangular frame from the contour image of the cap and stipe separated from the giant puffball mushroom, and then connect the center points of the minimum rectangular frames of the cap and stipe to obtain the connection vector;

[0025] Calculate the angle between the connecting vector and the vertical vector;

[0026] If the judgment result indicates that the line connecting the center points of the smallest rectangular frame of the cap and the stipe of the Stropharia cap is perpendicular to the horizontal line, it is judged to be vertical, and the next step is directly performed;

[0027] If the judgment result indicates that the line connecting the center points of the minimum rectangular frame of the cap and the stipe of the Stropharia cap is not perpendicular to the horizontal line, it is judged that it is not vertical and needs to be rotated.

[0028] Optionally, the line connecting the center points of the minimum rectangular frame of the cap and the stipe of the Stropharia cap is not perpendicular to the horizontal line, and needs to be rotated, specifically including:

[0029] According to the minimum rectangular frame information of the obtained contour images of the cap and stipe of the Stropharia cap, the coordinates of the center point and the rotation point of the minimum rectangular frame of the cap and stipe are calculated;

[0030] Connecting the center points of the smallest rectangular frame of the cap and the stem of the Stropharia cap, calculating the angle between the connecting vector and the vertical vector to obtain the rotation angle;

[0031] Obtain a rotation matrix according to the rotation point and the rotation angle;

[0032] According to the rotation matrix, the coordinates of each pixel point of the original image are affine transformed to obtain the coordinates of each pixel point of the cap and stipe of the giant puffball mushroom rotated to the vertical state, and finally the contour image of the cap and stipe of the giant puffball mushroom rotated to the vertical state is obtained.

[0033] Optionally, the method further includes

[0034] The PC communication module is connected to the single-chip microcomputer to send the grade information of the giant puffball mushroom that is about to enter the grading system to the single-chip microcomputer.

[0035] Optionally, the method further includes

[0036] The single chip microcomputer controls the opening of the air pipes of the corresponding level by outputting high and low levels according to the level information of the stropharia, so as to blow the stropharia into different collection frames according to the predicted level.

[0037] Optionally, the method further includes

[0038] Determine whether the corresponding level of air pipe is open to blow the giant stropharia into the corresponding collection frame:

[0039] After blowing through the main air blowing pipe, if the photoelectric sensor corresponding to the supplementary air blowing pipe still senses the stropharia, it means that the main air blowing pipe has not blown the stropharia into the corresponding collection frame. In this case, the supplementary air blowing pipe is opened to supplement the stropharia. The air pressure of the supplementary air blowing pipe is higher than that of the main air blowing pipe. At the same time, the lever is opened and quickly rotated to a state of about 30° with the protective wall of the long conveyor belt. Under the action of the lever and the supplementary air blowing pipe, the stropharia is ensured to fall smoothly into the corresponding collection frame.

[0040] In order to achieve the purpose of the invention, the present invention also provides the following scheme:

[0041] An automatic grading system for Stropharia rugosa, comprising a feeding system, a transmission system, an image acquisition visual system and a grading system;

[0042] The feeding system is used to sequentially transport the Stropharia to be graded to the starting end of the transmission system;

[0043] The transmission system is used to transmit the stropharia that are discharged in sequence from a single channel to the working range of the image acquisition visual system, and to move the stropharia that have been identified out of the working range of the image acquisition visual system;

[0044] The image acquisition visual system is used to extract and analyze the information of the giant puffball mushroom, and transmit the acquired information to the grading system, wherein the information includes the information of the height and diameter of the giant puffball mushroom cap, the length and diameter of the stipe of the giant puffball mushroom, and the appearance color of the giant puffball mushroom, which comprehensively reflects the grade of the giant puffball mushroom;

[0045] The grading system is used to grade the Stropharia cap mushrooms according to the information conveyed by the image acquisition visual system and the automatic grading method for Stropharia cap mushrooms described in any of the aforementioned technical solutions.

[0046] Optionally, the feeding system includes a vibrating plate;

[0047] The vibration plate includes a controller, a feeding spiral ladder, a vibrator, a stropharia lever, and a feeding spiral ladder;

[0048] The controller is used to control the vibration frequency, amplitude, movement mode and other parameters of the vibrator;

[0049] The vibrator transports the giant stropharia on the vibration plate to the transmission system through vibration force;

[0050] The feeding spiral ladder is used to carry the giant stropharia, so that it can produce the effects of moving single-row feeding and sorting;

[0051] The feeding spiral staircase is used to smoothly convey the stropharia that passes through the feeding spiral staircase to the transmission system;

[0052] The stropharia pushing rod is used to push down the stropharia standing upright on the feeding spiral ladder so that the stropharia can be laid flat for transmission.

[0053] Optionally, the vibrator is installed at the bottom center of the vibration plate, which generates vibration force through electromagnetic principles, so that the large stropharia piled at the bottom rises from bottom to top along the spiral staircase and realizes single-row loading and is transported to the transmission system.

[0054] Optionally, the loading spiral staircase rises in a spiral from the bottom of the vibrating plate along the wall surface of the vibrating plate to the inlet end of the feeding spiral staircase.

[0055] Optionally, the stropharia pushing rod is arranged between the first and second spiral stairs of the feeding spiral stairs and close to the second spiral stairs. The stropharia pushing rod is fixedly connected to the wall of the vibration plate as a whole and is used to push down the upright stropharia to make it lie flat for transmission.

[0056] Optionally, the feeding spiral ladder and the loading spiral ladder are both provided with shallow grooves;

[0057] The shallow grooves are used to ensure that there is a certain distance between adjacent stropharia, so as to carry out orderly transmission.

[0058] Optionally, the transmission system includes an elongated conveyor belt, which is made of a blue PU food-grade conveyor belt and is used to transmit the Stropharia officinalis transmitted sequentially through the feeding system to the working range of the image acquisition vision system and the grading system.

[0059] Optionally, the image acquisition visual system includes a dark box sensor, an industrial camera and a camera beam;

[0060] The dark box sensor is fixed above the transmission system;

[0061] The industrial camera and the camera beam are fixed on the upper end of the dark box sensor, and the dark box sensor and the industrial camera are located on the same vertical line. When the industrial camera is sensed to be directly below the giant stropharia, it is controlled to automatically capture the required image.

[0062] Optionally, the grading system includes a single chip microcomputer and a plurality of groups of air pipes, photoelectric sensors, a collection frame, and a lever connected to the single chip microcomputer for communication;

[0063] The air pipe is located at one side of the transmission system, the photoelectric sensor is installed near the air pipe to check whether the stropharia with corresponding grade information has arrived, and the collection frame is used to collect the stropharia with corresponding grade dropped from different positions of the transmission system;

[0064] When a photoelectric sensor senses that the giant stropharia with corresponding level information has not been blown off, the corresponding air pipe and lever action will be used for supplementary blowing.

[0065] Optionally, the single-chip microcomputer uses an STM32 single-chip microcomputer, which is communicatively connected to a PC communication module, and the latter sends the grade information of the giant puffball mushroom that is about to enter the grading system to the single-chip microcomputer. After receiving the grade information, the single-chip microcomputer puts the grade information in a grade queue. When the giant puffball mushroom reaches the corresponding photoelectric sensor, the single-chip microcomputer uses a digital output port to control the switch of the corresponding air pipe to perform automatic grading of the quality of the giant puffball mushroom.

[0066] Optionally, four air pipes are provided, two of which are used as main air pipes for controlling the grading of medium and small stropharia respectively, and the remaining two air pipes are used as supplementary air pipes for controlling the grading of medium and small stropharia. If the corresponding photoelectric sensor senses that the stropharia has not been blown off, the corresponding supplementary air pipe and the lever are opened at the same time, the air pressure in the supplementary air pipe is higher than that in the main air pipe, and after the lever is opened, it rotates quickly to a state of about 30° with the protective wall of the long conveyor belt to complete the supplementary blowing action.

[0067] Optionally, for large stropharia, they are directly transported to a collecting frame at the end of the long conveyor belt along with the conveying system.

[0068] Optionally, the grading system further comprises an air pump tank, which is used to contain and compress the gas and discharge the compressed gas through the air pipe.

[0069] Optionally, the grading system further comprises a tracheal stent, the trachea is fixed on the tracheal stent, the trachea stent is fixed on one side of the elongated conveyor belt, the photoelectric sensor is fixed near the tracheal stent, and the lever is fixed on a side opposite to the tracheal stent.

[0070] Compared with the prior art, the present invention has the following technical effects:

[0071] 1. The present invention uses a convolutional neural model and an image algorithm to grade the quality of Stropharia rugosa. This method is simple, effective and efficient, has a very high accuracy rate in identifying Stropharia rugosa, and ensures the accuracy of automatic grading.

[0072] 2. The present invention uses a vibration plate with a lever to load the stropharia, and can put a plurality of stropharia at a time without manual placement and greatly reduces the labor force. The added stropharia lever is made of flexible material, which can make the stropharia lie flat and enter the transmission system in an orderly manner while ensuring the quality of the stropharia.

[0073] 3. The present invention uses air pipes to automatically grade the giant puffball. The greatest feature of the air pipes is that they can be opened and closed, and the air pressure can also be adjusted manually. Therefore, the present invention can also be used on production lines that require a higher conveyor belt speed. At the same time, the added supplementary air pipes and levers further ensure the reliability and feasibility of the automatic grading system. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments. The drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can be obtained based on these drawings without any labor and creativity.

[0075] Figure 1 An overall schematic diagram of an automatic grading system for Stropharia rugosa provided by the present invention;

[0076] Figure 2 An overall schematic diagram of a Stropharia cap mushroom feeding system provided by the present invention;

[0077] Figure 3 A flow chart of a method for automatic grading of Stropharia rugosa provided by the present invention;

[0078] Figure 4 A complete flow chart of an automatic grading method for Stropharia rugosa provided by the present invention;

[0079] Figure 5 A schematic diagram of the grading standards of the automatic grading method of Stropharia officinalis provided by the present invention;

[0080] Figure 6 The image of the giant puffball mushroom processed by the segmentation model provided by the present invention;

[0081] Figure 7 The segmentation mask images of the cap and stipe with different colors provided by the present invention;

[0082] Figure 8A binary image of the Stropharia rugosa provided by the present invention;

[0083] Fig. 9 The present invention provides an image of a cap and a stem with a minimum rectangular frame rotated to a vertical state;

[0084] Description of reference numerals:

[0085] 1-vibration plate, 2-dark box sensor, 3 / 5 / 10 / 12-photoelectric sensor, 4-main air blowing pipe of small stropharia, 6-supplementary air blowing pipe of small stropharia, 7-air pump tank, 8-collection frame, 9-supplementary air blowing pipe of medium stropharia, 11-main air blowing pipe of medium stropharia, 13-push rod, 14-long conveyor belt, 15-PC, 16-industrial camera and camera beam. Specific implementation methods

[0086] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0087] It should be noted that the embodiments described are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by researchers of ordinary skill in the art without creative work based on the embodiments of the present invention are within the scope of protection of the present invention.

[0088] It should be noted that, for the sake of ease of description, the present invention takes the quality grading of Pleurotus ostreatus as an example to explain the present technical solution in detail. It is obvious to those skilled in the art that other mushrooms with similar appearance or the same appearance characteristics should also be applicable to the present automatic grading method and system.

[0089] In order to make the above-mentioned objects, features and advantages of the present invention clearer and easier to understand, the present invention will be further explained in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0090] Figure 1 The overall structure diagram of an automatic grading system for Stropharia rugosa provided by an embodiment of the present invention is as follows: Figure 1 As shown, the present invention provides a machine vision-based automatic grading system for Stropharia rugosa, comprising: a feeding system, a transmission system, an image acquisition vision system and a grading system.

[0091] The feeding system includes a vibrating plate 1, and the feeding system is located on one side of the long conveyor belt 14. Figure 2As shown, the vibration plate 1 includes a controller 101, a chassis 102, a feeding spiral staircase 103, a vibrator 104, a stropharia lever 105, a feeding spiral staircase 106 and a shallow groove 107. The controller 101 is used to control the vibration frequency, amplitude, movement mode and other parameters of the vibrator 104; the chassis 102 supports the entire vibration plate 1 structure and carries the vibration mechanical parts; the stropharia lever 105 is arranged between the first and second spiral staircases of the feeding spiral staircase 106 and close to the second spiral staircase. The two levers are welded to the wall of the vibration plate 1 and placed on opposite sides to knock down the upright stropharia so that it can lie flat for transmission; the feeding spiral staircase 106 is a spiral staircase that starts from the bottom of the vibration plate 1 and moves along the wall of the vibration plate 1. The entrance of the feeding spiral staircase 103 rises to carry the stropharia, so that it can be moved and sorted; the feeding spiral staircase 103 is used to smoothly transport the stropharia through the loading spiral staircase 106 to the long conveyor belt 14; the vibrator 104 is installed at the center of the bottom of the vibration plate 1, and generates vibration force through the electromagnetic principle, so that the stropharia stacked at the bottom rises from bottom to top along the spiral staircase and realizes single-row loading and is transported to the transmission system; the shallow groove 107 is used to ensure that there is a certain distance between adjacent stropharia, so as to carry out orderly transmission.

[0092] In practical applications, the controller 101 can adjust the motion parameters according to the actual operation conditions to achieve the control of the uniform and stable upward movement and sorting of the spherical mushrooms, reduce the possibility of reducing economic benefits, and will not cause blockage of the upward movement channel or damage to the fungus. The controller 101 can adjust the movement speed of the spherical mushrooms on the feeding spiral ladder 106 within the range of 4mm / s-16mm / s by increasing or decreasing the thickness of the fastening elastic element or adjusting the tightness of the fastening elastic element; the controller 101 can adjust the gap of the electromagnet coil by adjusting the knob to keep the normal gap between the electromagnet and the armature within the range of 0.5mm to 1.2mm, and control the amplitude of the vibrator 104 to limit the distance of the spherical mushrooms jumping up and down on the feeding spiral ladder 106 within the range of 0.1mm-0.8mm and reduce noise; the controller 101 can also control the half-wave and full-wave adjustment of the vibration frequency by switching the switch. The spherical mushroom lever 105 can prevent the spherical mushrooms from standing upright, knock down the upright spherical mushrooms, and at the same time do not affect the passage of the spherical mushrooms lying normally. Since the vertical height of the stropharia lever 105 is low, when the upright stropharia is knocked down by the stropharia lever 105, the mushroom body will not be damaged. The shallow groove 107 is a shallow bottom arc surface groove, and the maximum depth is 1 / 4 of the height of the stropharia lying flat, which can ensure that the stropharia will not be blocked in the groove, and a certain gap can also be guaranteed. A thin sponge can be placed at the bottom of the vibration plate 1 and the spiral ladder to further reduce the buffer, prevent damage to the mushroom body, and reduce the possibility of reducing economic benefits.

[0093] The transmission system includes a long transmission belt 14, which is made of a blue PU food-grade transmission belt and is used to transmit the Stropharia officinalis transmitted in sequence through the feeding system to the working range of the image acquisition vision system and the grading system.

[0094] The image acquisition visual system includes a dark box sensor 2, an industrial camera and a camera beam 16. The dark box sensor 2 is fixed above the front end of the long conveyor belt 14; the industrial camera and the camera beam 16 are fixed on the upper end of the dark box sensor 2. The dark box sensor 2 and the central axis of the industrial camera are located on the same vertical line. When it is sensed that the giant puffball is directly below the industrial camera, it is controlled to automatically capture the required image.

[0095] The grading system includes an air pump tank 7, an air pipe, a lever 13, a photoelectric sensor 3 / 5 / 10 / 12, a collection frame 8 and other auxiliary components. The air pump tank 7 is used to contain and compress the air and discharge the compressed air through the air pipe; the air pipe is fixed on one side of the long conveyor belt 14; the photoelectric sensor 3 / 5 / 10 / 12 is fixed near the corresponding air pipe to check whether the stropharia with corresponding grade information has arrived or is blown normally. The grading system is composed of two major parts: a control component and a collection component; the control component includes a PC communication module, an STM32 single-chip microcomputer, and two groups of photoelectric sensors 3 / 5 / 10 / 12, air pipes, and a lever 13 that are connected to the control component for communication. The air pipes and photoelectric sensors 3 / 5 / 10 / 12 are grouped in pairs. When a photoelectric sensor 3 / 5 / 10 / 12 senses that the stropharia with corresponding grade information is not blown off by the main air pipe, the supplementary air pipe and the lever 13 are activated for supplementary blowing; the PC communication module is used to transmit the grade information predicted by the automatic grading method to the STM32 single-chip microcomputer, and the STM32 single-chip microcomputer controls the opening and closing of the corresponding air pipe to complete the quality grading operation of the stropharia. The collection component includes a plurality of collection frames 8, which are used to collect the stropharia of corresponding grades that fall from different positions of the long conveyor belt 14.

[0096] The grading system also includes a tracheal stent, the trachea is fixed on the tracheal stent, the trachea is fixed on one side of the elongated conveyor belt 14, the photoelectric sensor 3 / 5 / 10 / 12 is fixed near the tracheal stent, and the lever 13 is fixed on the side opposite to the tracheal stent.

[0097] After the work starts, the stropharia are smoothly transmitted in sequence through the feeding system. When the stropharia are transported to the dark box sensor 2 via the long conveyor belt 14, the industrial camera is controlled by the signal output by the dark box sensor 2 to take pictures, and the captured stropharia images are transmitted to the PC 15. The PC 15 will grade the captured stropharia images according to the grading method. After receiving the grade information, the single chip microcomputer stores the grade information in the grade queue. When the stropharia arrives at the corresponding photoelectric sensor 3 / 5 / 10 / 12, the single chip microcomputer uses the digital output port to control the switch of the corresponding air pipe. When the digital output port is set to a high level, the corresponding air pipe is opened, otherwise, the air pipe is closed. There are four air pipes in total, two of which (main air blowing pipe 4 for small stropharia and main air blowing pipe 11 for medium stropharia) respectively control the grading of medium and small stropharia. For large stropharia, they are directly transported to the collection frame 8 at the end of the long conveyor belt 14 along with the transmission system; the other two air pipes (supplementary air blowing pipe 6 for small stropharia and supplementary air blowing pipe 9 for medium stropharia) are used as supplementary blowing devices for controlling the grading of medium and small stropharia. If the corresponding photoelectric sensor 3 / 5 / 10 / 12 senses that the stropharia is not blown off by the main air blowing pipe, the corresponding supplementary air pipe and the lever 13 are opened at the same time, and the air pressure of the supplementary air pipe is higher than that of the main air blowing pipe. After the lever 13 is opened, it rotates quickly to a state of about 30° with the protective wall of the long conveyor belt 14 to ensure that the stropharia stably falls into the correct collection frame 8 to complete the supplementary blowing action.

[0098] In actual application, in order to ensure that the surface of the stropharia is not damaged, the lever 13 is made of rubber material, and the lever 13 is generally kept closed. In order to ensure that the stropharia after classification can fall into the collection frame 8 without damage, a drop test is carried out to determine that the distance from the center point of the long conveyor belt 14 to the lowest point of the collection frame 8 should be kept within the range of 145 to 285 mm, and a layer of sponge pads needs to be laid at the bottom of the collection frame 8 to further increase protection measures. The collection frame 8 can accommodate up to 50 stropharia at a time, and needs to be manually emptied when the stropharia is full.

[0099] according to Figure 3 , Figure 4 As shown, the embodiment of the present invention further provides an automatic grading method for Stropharia capitis, which is applied to the automatic grading system for Stropharia capitis described above, and the method comprises:

[0100] Step 101: Acquire an image of the giant puffball mushroom on a transmission system.

[0101] Step 102: Perform image distortion correction on the acquired stropharia image. In order to accurately calculate the actual pixel size of stropharia, the intrinsic parameters and distortion coefficients of the industrial camera are obtained by completing camera calibration, and the image distortion introduced by the camera is corrected by changing the two parameters. Specifically, it includes:

[0102] The coordinates (u, v) of the pixel coordinate system of the two-dimensional image of the giant mushroom are converted into the coordinates (x, y) of the giant mushroom in the image coordinate system according to the conversion formula, wherein the origin of the pixel coordinate system is moved to the center of the image as the origin of the image coordinate system. The conversion formula is:

[0103]

[0104] Among them, the pixel coordinates of the center of the image are (u0, v0), and the physical size of each pixel of the industrial camera photosensitive device is dx*dy;

[0105] According to the conversion formula, the coordinates (x, y) of the image coordinate system of the giant mushroom are converted into the coordinates (X, Y) of the camera coordinate system with the center of the fixed industrial camera as the origin. c ,Y c ,Z c ), the conversion formula is:

[0106]

[0107] Among them, f is the focal length of the camera, that is, the distance between the origin of the camera coordinate system and the image coordinate system, x and y represent the horizontal and vertical pixel numbers of the giant mushroom in the image coordinate system, and Z c is the distance from the Stropharia to the optical center;

[0108] According to the conversion formula, the coordinates in the camera coordinate system are converted into the coordinates in the world coordinate system with the dark box base as the origin (X w ,Y w ,Z w ), the conversion formula is:

[0109]

[0110] Among them, u and v are the center pixel coordinates of the image in the pixel coordinate system; R is a 3×3 rotation matrix that describes the rotation transformation of the camera; T is the translation vector that describes the translation of the camera in the x, y, and z directions;

[0111] According to the above principle, the intrinsic and extrinsic parameters and distortion parameters of industrial cameras can be calibrated;

[0112] The internal parameter matrix is ​​α1 and the external parameter matrix is ​​α2 respectively:

[0113]

[0114]

[0115] According to the internal and external parameters of the camera, the image pixel coordinates (x, y) after camera distortion correction are obtained as follows:

[0116] (x,y)=α1×α2×(X c ,Y c )

[0117] By completing the calibration, the intrinsic parameters and distortion coefficients of the industrial camera are obtained. The two parameters can be used to transform and correct the newly taken Stropharia images, and the corrected Stropharia images can be used for training and image processing.

[0118] Step 103: The distortion-corrected Stropharia images are further organized into corresponding data sets. The caps and stipes of Stropharia are manually labeled using different colors according to different levels to create a data set that meets the requirements. The data set is then passed into the convolutional neural network segmentation model, which can be used to output segmentation mask images with different colors of caps and stipes after training, such as Figure 6 shown.

[0119] Step 104: In order to accurately classify the Stropharia cap according to the classification standard, the extracted segmented images of the Stropharia cap and the stipe are accurately classified according to the prescribed classification standard through an image processing algorithm, specifically including:

[0120] The segmentation mask images of the stropharia with different colors of caps and stipes are color separated. The data set applied to the convolutional neural network segmentation model contains three levels of stropharia, namely: first-class mushrooms, second-class mushrooms, and third-class mushrooms. The colors of the segmentation mask images of the three manually labeled stropharia are different. In order to make the method adaptable to the three levels of stropharia, the caps and stipes of the stropharia are processed with segmentation masks of different colors according to the convolutional neural segmentation model, and the segmentation mask images of the caps and stipes of the stropharia with different color segmentation masks are processed by color separation, and the images are split into three channels of b, g, and r;

[0121] The three channels b, g, and r obtained after color separation are fused to obtain multiple color channels, and finally the segmentation mask images of the caps and stipes with different colors in the image can be completely extracted, such as Figure 7 As shown;

[0122] The segmentation mask images of the caps and stipes with different colors are binarized, and the points with non-0 pixel values ​​in all pixels are set to 1, so that the binarized images of the caps and stipes can be obtained respectively, as shown in FIG. Figure 8 As shown;

[0123] Extracting contour features from the binary images of the cap and stipe respectively to obtain corresponding contour images and feature information thereof;

[0124] According to the outline image of the cap and stipe of the Stropharia capifolia, the minimum rectangular frame that can enclose the outline of the cap and stipe of the Stropharia capifolia is found, and the upper left corner coordinates (x0, y0), as well as the width w0 and height h0 of the minimum rectangular frame are obtained respectively.

[0125] The diameter and height of the cap of the stropharia and the diameter and length of the stipe are analogized according to the length and width of the minimum circumscribed rectangle that can enclose the outline of the cap and stipe of the stropharia. However, in the actual process, the image acquisition visual system may capture stropharia at different rotation angles. The minimum circumscribed rectangles at different angles may cause the diameter and length of the cap and stipe of the stropharia to be inverted. In order to unify the acquisition of the diameter and height of the cap and the diameter and length of the stipe, it is necessary to rotate all the stropharia images to a vertical state in advance. The specific process is as follows:

[0126] According to the information of the minimum rectangular frame surrounding the cap and stipe of the Stropharia cap, the coordinates of the center point of the minimum rectangular frame surrounding the cap are calculated as And the coordinates of the center point of the minimum rectangular frame surrounding the stem are

[0127] The vector connecting the center points of the cap and the stipe of the Stropharia cap is called the connection vector k:

[0128]

[0129] Calculate the angle between the line vector k and the vertical direction. Assuming that the vertical vector a is <0,1>, use the vector dot product to calculate the rotation angle θ:

[0130]

[0131] Among them, |k| and |a| are the moduli of the corresponding vectors respectively;

[0132] According to the coordinates of the center point of the minimum rectangular frame surrounding the cap And the coordinates of the center point of the minimum rectangular frame surrounding the stem So we can get the coordinates of the rotation point as Marked as (x center ,y center );

[0133] From the above rotation angle θ and the rotation point coordinates (x center ,y center ), the rotation matrix γ is:

[0134]

[0135] Among them, the first two columns of the above 2×3 rotation matrix γ represent rotation transformations, and the last column represents translation transformation;

[0136] According to the rotation matrix γ, the coordinates (x, y) of each pixel point in the original image can be converted into the image of the cap and stem of the giant puffball mushroom by affine transformation and rotated into the coordinates (x ′ ,y ′ ),like Fig. 9 As shown in the figure, the coordinates of each pixel after affine transformation (x ′ ,y ′ )for:

[0137]

[0138] Calculate the number of pixels corresponding to the length and width of the minimum rectangular frame that surrounds the cap and stipe of the Stropharia cap rotated to a vertical state, and record them as n1, m1, n2, and m2 respectively;

[0139] According to the number of pixels calculated, the ratio of diameter to height of the cap (RDHP) and the ratio of length to diameter of the stipe (RLDS) are calculated respectively, and the formulas are as follows:

[0140]

[0141]

[0142] According to the ratio of diameter to height of the cap (RDHP) and the ratio of length to diameter of the stipe (RLDS), a detection model of the ratio of diameter to width of the cap (RDHP) and the ratio of length to diameter of the stipe (RLDS) was established, and the grading of the Stropharia cap was carried out according to the grading standard, and the grading of the Stropharia cap was carried out according to the grading standard. Figure 5 , specifically including:

[0143] If the RDHP is within the range of 0.0 to 2.5 and the RLDS is within the range of 0.0 to 3.0, the output is a first-level mushroom;

[0144] If the RDHP is within the range of 2.5 to 4.0 and the RLDS is within the range of 3.0 to 4.5, the output is a secondary mushroom;

[0145] If the RDHP is greater than 4.0 and the RLDS is greater than 4.5, the output is a level 3 mushroom;

[0146] If the RDHP is within one of the above ranges, but the RLDS is not within the range corresponding to the above categories, it shall be classified according to the level to which the RLDS belongs.

[0147] In actual application, the stropharia described in step 101 is produced in Dahuashan Town, Pinggu District, Beijing. In order to prevent the stropharia from being damaged during transportation, the harvested stropharia will be packed in cartons. A large number of cartons will be placed in vehicles that can adjust the temperature and humidity and transported to the grading factory. The temperature is maintained between 0 degrees Celsius and 4 degrees Celsius, and the relative humidity is maintained at 90%. The stropharia of different grades transported to the grading factory are photographed to obtain experimental samples for experiments.

[0148] In practical applications, the image of the giant puffball taken by the industrial camera described in step 102 has optical distortion. Different cameras have different intrinsic and extrinsic parameters. Before correcting the captured image, the intrinsic and extrinsic parameters of the camera need to be determined.

[0149] In practical applications, the convolutional neural network segmentation model described in step 103 needs to be trained in advance, specifically including:

[0150] The image acquisition visual system of the stropharia is used to collect images of stropharia of different levels. A total of 420 stropharia of different levels are randomly selected, with 140 stropharia of each level. The 420 stropharia are placed on the vibration plate 1. The stropharia will be imaged sequentially. When the stropharia passes through the dark box sensor 2, the image acquisition visual system will automatically acquire the image of the stropharia until the image acquisition of 420 stropharia is completed.

[0151] Manually label the images of Stropharia. Use the labelimg tool to label the 420 images of Stropharia according to three levels: first-class mushrooms, second-class mushrooms, and third-class mushrooms. Each Stropharia image will be labeled with the cap and stipe areas, and the corresponding label files will be saved.

[0152] Create a dataset for model training. Randomly select 100 images (300 images in total) and the corresponding label files in each level as the training set, then randomly select 20 images (60 images in total) and the corresponding label files as the validation set, and the remaining 60 images and the corresponding label files as the test set;

[0153] Training of the segmentation model. The generated dataset is fed into the convolutional neural network segmentation model for training;

[0154] Generate a segmentation model. Usually, after the model is trained, it will generate an optimal weight and the weight of the last round. The optimal weight is selected as the weight of the model to generate a convolutional neural network segmentation model.

[0155] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the concept of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. An automatic grading method for Stropharia rugosotis, characterized in that: The method comprises Get the image of Stropharia rugosotis; Perform image correction on the acquired Stropharia rugosa images; After annotating the corrected Stropharia images, a corresponding data set is created, which is then passed into the convolutional neural model to obtain the segmentation mask images of the Stropharia cap and stipe; Performing color separation according to the segmentation mask image of the cap and stipe of the Stropharia capifolia to obtain segmentation images of the cap and stipe with different colors; Performing image processing on the segmented images of the caps and stipes with different colors to obtain contour images of the separated caps and stipes of the Stropharia capifolia and their feature information, and the minimum circumscribed rectangle of the separated caps and stipes of the Stropharia capifolia and their coordinate information; According to the minimum circumscribed rectangle of the separated cap and stipe of the Stropharia capifolia and its coordinate information, whether the center line connecting the minimum circumscribed rectangle of the cap and stipe is vertical is judged to determine whether to rotate the image of the separated cap and stipe of the Stropharia capifolia, and finally obtain the vertical contour image of the separated cap and stipe of the Stropharia capifolia and its characteristic information; According to the outline image of the vertical Stropharia cap and stipe separated and its characteristic information and the minimum circumscribed rectangle and its coordinate information, a detection model of the cap diameter-width ratio and the stipe diameter-length ratio is finally established, so as to determine the grade of Stropharia cap according to the grading standard.

2. The automatic grading method of Stropharia officinalis according to claim 1, characterized in that: The segmented images of the caps and stipes with different colors are processed to obtain the minimum circumscribed rectangles of the separated caps and stipes of the Stropharia capifolia and their coordinate information, specifically including: Binarization is performed on the segmentation mask images of the caps and stipes with different colors, and all non-0 pixels are set to 1 to obtain binary images of the caps and stipes; Extracting contour features of the binary images of the cap and stipe respectively to obtain contour images of the cap and stipe and their characteristic information; The minimum rectangular frame is obtained according to the outline image of the cap and the stipe, and the minimum circumscribed rectangle surrounding the outline image of the cap and the stipe of the Stropharia capensis and its coordinate information are obtained.

3. The automatic grading method of Stropharia officinalis according to claim 1, characterized in that: The step of judging whether the center line connecting the minimum circumscribed rectangle of the cap and the stipe is vertical to determine whether the images of the cap and the stipe separated from the Stropharia cap are rotated specifically includes: Extract the minimum rectangular frame from the contour image of the cap and stipe separated from the giant puffball mushroom, and then connect the center points of the minimum rectangular frames of the cap and stipe to obtain the connection vector; Calculate the angle between the connecting vector and the vertical vector; If the judgment result indicates that the line connecting the center points of the smallest rectangular frame of the cap and the stipe of the Stropharia cap is perpendicular to the horizontal line, it is judged to be vertical, and the next step is directly performed; If the judgment result indicates that the line connecting the center points of the minimum rectangular frame of the cap and the stipe of the Stropharia cap is not perpendicular to the horizontal line, it is judged that it is not vertical and needs to be rotated.

4. The automatic grading method of Stropharia officinalis according to claim 3, characterized in that: The line connecting the center points of the minimum rectangular frame of the Stropharia cap and the stipe is not perpendicular to the horizontal line, and needs to be rotated, specifically including: According to the minimum rectangular frame information of the obtained contour images of the cap and stipe of the Stropharia cap, the coordinates of the center point and the rotation point of the minimum rectangular frame of the cap and stipe are calculated; Connecting the center points of the smallest rectangular frame of the cap and the stem of the Stropharia cap, calculating the angle between the connecting vector and the vertical vector to obtain the rotation angle; Obtain a rotation matrix according to the rotation point and the rotation angle; According to the rotation matrix, the coordinates of each pixel point of the original image are affine transformed to obtain the coordinates of each pixel point of the cap and stipe of the giant puffball mushroom rotated to the vertical state, and finally the contour image of the cap and stipe of the giant puffball mushroom rotated to the vertical state is obtained.

5. The automatic grading method of Stropharia officinalis according to claim 1, characterized in that: The method further comprises According to the grade information of the giant puffball mushroom, the air pipe of the corresponding grade is controlled to open by outputting high and low levels, so that the giant puffball mushroom is blown into different collection frames according to the predicted grade.

6. The automatic grading method of Stropharia officinalis according to claim 5, characterized in that: The method further comprises Determine whether the corresponding level of air pipe is open to blow the giant stropharia into the corresponding collection frame: After blowing through the main air blowing pipe, if the photoelectric sensor corresponding to the supplementary air blowing pipe still senses the stropharia, it means that the main air blowing pipe has not blown the stropharia into the corresponding collection frame. In this case, the supplementary air blowing pipe is opened to supplement the stropharia. The air pressure of the supplementary air blowing pipe is higher than that of the main air blowing pipe. At the same time, the lever is opened and quickly rotated to a 30° angle with the protective wall of the long conveyor belt. Under the action of the lever and the supplementary air blowing pipe, the stropharia is ensured to fall smoothly into the corresponding collection frame.

7. An automatic grading system for Stropharia rugosotis, characterized by: Including feeding system, transmission system, image acquisition vision system and grading system; The feeding system is used to sequentially transport the Stropharia to be graded to the starting end of the transmission system; The transmission system is used to transmit the stropharia that are discharged in sequence from a single channel to the working range of the image acquisition visual system, and to move the stropharia that have been identified out of the working range of the image acquisition visual system; The image acquisition visual system is used to extract and analyze the information of the giant puffball mushroom, and transmit the acquired information to the grading system, wherein the information includes the cap height and diameter information of the giant puffball mushroom, the stem length and diameter information of the giant puffball mushroom, and the appearance color of the giant puffball mushroom, which comprehensively reflects the grade of the giant puffball mushroom; The grading system is used to grade the Stropharia cap mushrooms according to the information conveyed by the image acquisition visual system and the automatic grading method for Stropharia cap mushrooms according to any one of claims 1 to 6.

8. The automatic grading system for Stropharia rugosa according to claim 7, characterized in that: The feeding system includes a vibrating plate; The vibration plate includes a controller, a feeding spiral ladder, a vibrator, a stropharia lever, and a feeding spiral ladder; The controller is used to control the vibration frequency, amplitude and movement mode of the vibrator; The vibrator transports the giant stropharia on the vibration plate to the transmission system through vibration force; The feeding spiral ladder is used to carry the giant stropharia, so that it can be fed and sorted in a moving single row; The feeding spiral staircase is used to smoothly convey the stropharia that passes through the feeding spiral staircase to the transmission system; The stropharia pushing rod is used to push down the stropharia standing upright on the feeding spiral ladder so that the stropharia can be laid flat for transmission.

9. The automatic grading system for Stropharia rugosa according to claim 7, characterized in that: The image acquisition visual system includes a dark box sensor, an industrial camera and a camera beam; The dark box sensor is fixed above the transmission system; The industrial camera and the camera beam are fixed on the upper end of the dark box sensor, and the dark box sensor and the industrial camera are located on the same vertical line. When the industrial camera is sensed to be directly below the giant stropharia, it is controlled to automatically capture the required image.

10. The automatic grading system for Stropharia rugosa according to claim 7, characterized in that: The grading system includes a single chip microcomputer and multiple groups of air pipes, photoelectric sensors, a collection frame, and a lever connected to the single chip microcomputer for communication; The air pipe is located at one side of the transmission system, the photoelectric sensor is installed near the air pipe to check whether the stropharia with corresponding grade information has arrived, and the collection frame is used to collect the stropharia with corresponding grade dropped from different positions of the transmission system; When a photoelectric sensor senses that the giant stropharia with corresponding level information has not been blown off, the corresponding air pipe and lever action will be used for supplementary blowing.

Images