Method and device for detecting uniformity of Flammulina velutipes based on depth camera
The depth image of the enoki mushroom is obtained through a depth camera, and its average height and neatness are automatically calculated, which solves the problem of strong subjectivity of manual evaluation in the existing technology, realizes rapid and accurate detection of the neatness of the enoki mushroom, and improves the standardization of factory production level.
Patent Information
- Application Number
- CN202010704803.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-07-21
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2040-07-21
AI Technical Summary
In the prior art, the quality grading of enoki mushrooms relies on manual evaluation, is subjective and difficult to effectively detect neatness, which limits the standardization and accuracy of factory production of enoki mushrooms.
Depth camera is used to obtain depth images from directly above the enoki mushrooms, extract the average height and neatness of the bacteria clusters through image processing, and automatically detect them using depth cameras and computers.
The fast, objective and accurate evaluation of the neatness of enoki mushrooms has been achieved, and the standardization of factory production has been improved.
Smart Images

Figure CN111707194B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of agricultural technology, and in particular to a method and device for detecting uniformity of enoki mushrooms based on a depth camera. Background Art
[0002] Flammulina velutipes, with its smooth cap, crisp stem and delicious taste, is one of the famous edible mushrooms in ancient and modern times, and is also one of the most popular natural health foods in the market today. As the industrialization of Flammulina velutipes develops rapidly, the automatic detection and grading of its production and quality indicators are also receiving more and more attention.
[0003] The applicant has found that for Flammulina velutipes, the uniformity of the mushroom cluster is an important indicator for evaluating its grade. However, in the prior art, the standardization of Flammulina velutipes factory production is low, and its growth and picking time are judged by visual observation based on experience. The quality grading of Flammulina velutipes mostly relies on manual evaluation, which is time-consuming, labor-intensive and highly subjective. The objectivity and accuracy of the evaluation results are insufficient, which has become a bottleneck restricting the factory production and cultivation of Flammulina velutipes.
[0004] For example, the Chinese invention with publication number CN109815923A provides a method for sorting and identifying enoki mushroom heads based on LBP features and deep learning. This series of prior art proposes a enoki mushroom sorting and detection scheme based on image recognition. However, this series of schemes mostly detect the size and color of a single mushroom body, while for fungi such as enoki mushrooms that grow in clusters, it is obvious that neatness is an important evaluation index. If a color camera is used to obtain the neatness index of enoki mushrooms, two color images must be used to reconstruct its three-dimensional entity and then extract the height information. The steps are complicated and difficult to implement. Summary of the invention
[0005] In order to make up for the above-mentioned deficiencies in the prior art, the present invention obtains a depth image from the top of the entire cluster of Flammulina velutipes, and obtains the average height and uniformity of the entire cluster of Flammulina velutipes based on the single depth image. The detection method is simpler and more accurate.
[0006] In order to achieve the above-mentioned purpose, the present invention provides a method for detecting uniformity of Flammulina velutipes based on a depth camera, and its technical solution is as follows.
[0007] The method for detecting uniformity of Flammulina velutipes based on a depth camera comprises the following steps:
[0008] S1-Camera Calibration
[0009] The grayscale value of the image obtained from the depth camera is matched with the height of the stipe of Flammulina velutipes to obtain a calibration value;
[0010] S2-Image Acquisition
[0011] A depth camera is used to obtain a depth image from directly above the Flammulina velutipes cluster;
[0012] S3 - Image Processing
[0013] The background in the depth image is removed by using image filtering and segmentation methods, and the grayscale image of the bacterial cluster is extracted;
[0014] S4 - Get Metrics
[0015] According to the grayscale image of the mushroom cluster and the calibration value of step S1, the height of a single stipe in the Flammulina velutipes mushroom cluster is converted, and then the average height and uniformity of the stipes in the Flammulina velutipes mushroom cluster are calculated.
[0016] In order to help realize automated enoki mushroom uniformity detection, the present invention also proposes a enoki mushroom uniformity detection device based on a depth camera according to the above-mentioned enoki mushroom uniformity detection method based on a depth camera, and its technical solution is as follows.
[0017] A device for detecting uniformity of enoki mushrooms based on a depth camera, provided with a computer for executing the method for detecting uniformity of enoki mushrooms based on a depth camera;
[0018] There is also a Flammulina velutipes conveying device, which provides a frame, and a belt conveying device for conveying Flammulina velutipes mushroom clusters in a horizontal direction is provided on the top of the frame, and the Flammulina velutipes mushroom clusters are placed on the belt conveying device in a vertical position; a motor for driving the belt conveying device and a PLC control unit for controlling the start and stop of the motor are installed on the frame;
[0019] An image acquisition device is also provided, which provides a collection box installed on the upper end of the frame and straddling the belt conveyor device. The collection box has a front door and a rear door for the belt conveyor device to pass through in sequence. A depth camera is arranged on the top wall of the collection box and is centered above the belt conveyor device. The depth camera takes pictures of the enoki mushroom clusters on the belt conveyor device to obtain a depth image.
[0020] There is also an infrared photoelectric sensor installed on the side of the belt conveyor device to detect whether there are Flammulina velutipes clusters and whether the Flammulina velutipes clusters are transported directly under the depth camera in a direction perpendicular to the conveying direction;
[0021] An image storage device is also provided, which is connected to the depth camera and the computer respectively to store the depth image obtained by the depth camera, and transmit the depth image to the computer for uniformity detection of the enoki mushrooms;
[0022] Among them, the motor, depth camera, and infrared photoelectric sensor are electrically connected to the PLC control unit to realize PLC automatic control.
[0023] Compared with the prior art, the present invention has the following beneficial effects:
[0024] The height and uniformity indexes of the enoki mushrooms can be quickly obtained. Compared with the traditional method, the present invention evaluates the uniformity of the entire cluster of enoki mushrooms, and the evaluation result is more objective and accurate.
[0025] The present invention is further described below with reference to the accompanying drawings and specific implementation methods. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] Figure 1 It is a schematic flow chart of the method for detecting uniformity of enoki mushrooms based on a depth camera of the present invention.
[0027] Figure 2 It is a schematic diagram of a depth image in the method for detecting uniformity of Flammulina velutipes based on a depth camera of the present invention.
[0028] Figure 3 It is a schematic diagram of a grayscale image of a mushroom cluster in the method for detecting uniformity of Flammulina velutipes based on a depth camera of the present invention.
[0029] Figure 4 It is a structural schematic diagram of the Flammulina velutipes uniformity detection device based on a depth camera of the present invention.
[0030] Figure 5 It is a structural schematic diagram of the LED light source in the Flammulina velutipes uniformity detection device based on a depth camera of the present invention.
[0031] Figure 6 It is a principle block diagram of the Flammulina velutipes uniformity detection device based on a depth camera of the present invention.
[0032] Figure 7 It is a schematic diagram of the working process of the Flammulina velutipes uniformity detection device based on a depth camera of the present invention. DETAILED DESCRIPTION
[0033] like Figure 1 As shown, the method for detecting the uniformity of enoki mushrooms based on a depth camera includes the following steps:
[0034] S1-Camera Calibration
[0035] The grayscale value of the image obtained from the depth camera is matched with the height of the stipe of Flammulina velutipes to obtain a calibration value;
[0036] S2-Image Acquisition
[0037] A depth camera is used to obtain a depth image from directly above the Flammulina velutipes cluster; the obtained depth image is as follows: Figure 2 As shown;
[0038] S3 - Image Processing
[0039] Image filtering and segmentation methods are used to remove the background in the depth image and extract the grayscale image of the bacterial cluster. Figure 3 As shown;
[0040] S4 - Get Metrics
[0041] According to the grayscale image of the mushroom cluster and the calibration value of step S1, the height of a single stipe in the Flammulina velutipes mushroom cluster is converted, and then the average height and uniformity of the stipes in the Flammulina velutipes mushroom cluster are calculated.
[0042] The above-mentioned method for detecting the uniformity of enoki mushrooms based on a depth camera obtains a depth image from the top of the entire cluster of enoki mushrooms, and obtains the average height and uniformity of the entire cluster of enoki mushrooms according to a single depth image. The height and uniformity indicators of the enoki mushrooms can be quickly obtained. Compared with the traditional method, the present invention evaluates the uniformity of the entire cluster of enoki mushrooms, and the evaluation result is more objective and accurate.
[0043] Preferably, in step S1, the specific method of camera calibration is:
[0044] (1) Select the distance interval for measurement
[0045] It is planned to use 5cm as a measurement unit, and to conduct measurement and calibration at 8 calibration heights, namely, the ground, 5cm, 10cm, 15cm, 20cm, 25cm, 30cm, and 35cm above the ground;
[0046] (2) At each of the eight selected calibration heights, use a flat calibration plate with low reflectivity and place it on a plane parallel to the ground. Use the same procedure as step S2 to take pictures. Take five pictures at each position and store them by number.
[0047] (3) Read the five stored photos of the same calibrated height and perform image processing on the photos. Select the same area in the five photos and obtain the grayscale value of the area. Record the grayscale value of each photo and calculate the average of the five grayscale values. The average value is the grayscale value corresponding to the height in the depth image acquisition program. After performing the same processing on all eight calibrated heights, the regression equation for the relationship between grayscale value and height is obtained as follows:
[0048] y=-5.2881x+212.31
[0049] The precision of the regression equation is
[0050] R 2 =0.9972
[0051] Among them, x represents the height value and y represents the grayscale value.
[0052] Of course, in other implementations, measurement units of other sizes and other numbers of calibrated heights may be proposed, and other numbers of multiple photos may be taken at the same calibrated height.
[0053] Preferably, the calculation method of the regularity in step S4 is as follows:
[0054]
[0055] That is, the number of strains whose stipe heights in the whole cluster of Flammulina velutipes were within the range of ±5% of the average height was compared with the total number of stipes in the whole cluster of Flammulina velutipes, and the comparison results were presented in percentage.
[0056] According to the above-mentioned method for detecting the uniformity of Flammulina velutipes based on a depth camera, an experiment was conducted. 20 clusters of Flammulina velutipes were selected for testing. The samples were numbered 1-20. The calculation results of the uniformity are shown in the following table:
[0057]
[0058] According to the above-mentioned method for detecting uniformity of Flammulina velutipes based on a depth camera, the present invention proposes the following Figures 4 to 6 The enoki mushroom regularity detection device based on a depth camera shown is provided with a computer 3 for executing the enoki mushroom regularity detection method based on a depth camera;
[0059] A Flammulina velutipes conveying device is also provided, which provides a frame 11, and a belt conveying device 12 for conveying Flammulina velutipes mushroom clusters in a horizontal direction is provided on the top surface of the frame 11, and the Flammulina velutipes mushroom clusters are placed on the belt conveying device 12 in a vertical position; a motor 13 for driving the belt conveying device 12 and a PLC control unit 10 for controlling the start and stop of the motor 13 are installed on the frame 11;
[0060] An image acquisition device is also provided, providing a collection box 21 installed at the upper end of the frame 11 and crossing the belt conveyor 12. The collection box 21 has a front door 211 and a rear door 212 for the belt conveyor 12 to pass through in sequence. A depth camera 22 is arranged on the top wall of the collection box 21 and is centered above the belt conveyor 12. The depth camera 22 takes pictures of the enoki mushroom clusters on the belt conveyor 12 to obtain a depth image.
[0061] There is also an infrared photoelectric sensor 23 installed on the side of the belt conveyor 12 to detect whether there are mushroom clusters and whether the mushroom clusters are transported directly below the depth camera 22 in a direction perpendicular to the conveying direction;
[0062] An image storage device 20 is also provided, which is connected to the depth camera 22 and the computer 3 respectively by signals to store the depth image obtained by the depth camera 22, and transmit the depth image to the computer 3 for uniformity detection of the enoki mushrooms;
[0063] The motor 13, the depth camera 22, and the infrared photoelectric sensor 23 are electrically connected to the PLC control unit 10 to implement PLC automatic control.
[0064] The above-mentioned Flammulina velutipes uniformity detection device based on the depth camera 22 is used to calibrate the camera first, and the grayscale value of the image obtained from the depth camera 22 is matched with the height of the stipe of the Flammulina velutipes to obtain the calibration value; the whole cluster of Flammulina velutipes to be detected is placed on the belt conveyor 12, and the motor 13 drives the belt conveyor 12 to operate and transport the Flammulina velutipes to the collection box 21; when the infrared photoelectric sensor 23 detects that the Flammulina velutipes have arrived in the image collection area directly below the depth camera 22, a high level is sent to the PLC control unit, and the PLC control unit controls the motor 13 to pause so that the Flammulina velutipes are stationary in the image collection area; then the PLC controls The control unit triggers the depth camera 22 to obtain a depth image from directly above the Flammulina velutipes cluster; at the same time, the computer 3 processes the acquired depth image through the image storage device 20, removes the background in the depth image by using image filtering and segmentation methods, extracts the grayscale image of the cluster, and obtains the uniformity of the Flammulina velutipes; finally, the computer converts the height of a single stipe in the Flammulina velutipes cluster based on the grayscale image of the cluster and the pre-set calibration value, and then calculates the average height and uniformity of the stipes in the Flammulina velutipes cluster; after the current detection is completed, the PLC control unit controls the motor 13 to resume operation, completing the entire working process. The above workflow is as follows: Figure 7 shown.
[0065] Better, such as Figure 5 As shown, the interior of the collection box 21 is provided with LED light sources 24 for illuminating the interior of the collection box 21 on the top wall and the side wall. To facilitate the display of the assembly structure of the LED light source 24, Figure 5 The outer wall of the collection box 21 is hidden.
[0066] Better, such as Figure 5 As shown, the collection box 21 is a square box, and the LED light source 24 includes stepless dimming LED light strips arranged along the four vertical edges of the inner wall of the collection box 21, and the LED light source 24 also includes stepless dimming LED light strips arranged along the four horizontal edges of the top wall of the collection box 21.
[0067] Better, such as Figure 5 As shown, the stepless dimming LED light strips at the four vertical edges of the collection box 21 have their light-emitting surfaces facing the center line of the lens of the depth camera 22; the stepless dimming LED light strips at the four horizontal edges of the collection box 21 have their light-emitting surfaces facing the intersection of the center line of the lens of the depth camera 22 and the top surface of the belt conveyor 12.
[0068] Preferably, the depth camera 22 adopts the second generation Kinect camera released by Microsoft.
[0069] Better, such as Figure 4As shown, the infrared photoelectric sensor 23 is located opposite to the shooting range of the depth camera 22 to detect whether the Flammulina velutipes sent for inspection reaches the image acquisition area directly below the depth camera 22 .
[0070] Preferably, the infrared photoelectric sensor 23 is a diffuse reflection type infrared photoelectric switch of model E18-D80NK. The diffuse reflection type infrared photoelectric switch of model E18-D80NK is a photoelectric sensor integrating emission and reception, so it only needs to be installed on one side of the belt conveyor 12.
[0071] For those skilled in the art, the protection scope of the present invention is not limited to the details of the above-mentioned exemplary embodiments. Without departing from the spirit or basic characteristics of the present invention, all implementation methods with equivalent meanings and changes within the protection scope made by those skilled in the art based on the requirements of the present invention should be included in the present invention.
Claims
1. A method for detecting uniformity of enoki mushrooms based on a depth camera, characterized in that: The steps include: S1-Camera Calibration The grayscale value of the image obtained from the depth camera is matched with the height of the stipe of Flammulina velutipes to obtain a calibration value; S2-Image Acquisition A depth camera is used to obtain a depth image from directly above the Flammulina velutipes cluster; S3 - Image Processing The background in the depth image is removed by using image filtering and segmentation methods, and the grayscale image of the bacterial cluster is extracted; S4 - Get Metrics According to the grayscale image of the mushroom cluster and the calibration value of step S1, the height of a single stipe in the Flammulina velutipes mushroom cluster is converted, and then the average height and uniformity of the stipes in the Flammulina velutipes mushroom cluster are calculated; In step S1, the specific method of camera calibration is: (1) Select the distance interval for measurement It is planned to use 5cm as a measurement unit, and to carry out measurement and calibration at 8 calibration heights, namely, the ground, 5cm, 10cm, 15cm, 20cm, 25cm, 30cm, and 35cm above the ground; (2) At each of the eight selected calibration heights, use a flat calibration plate with low reflectivity and place it on a plane parallel to the ground. Use the same procedure as step S2 to take pictures. Take five pictures at each position and store them by number. (3) Read the five stored photos of the same calibrated height and perform image processing on the photos. Select the same area in the five photos and obtain the grayscale value of the area. Record the grayscale value of each photo and calculate the average of the five grayscale values. The average value is the grayscale value corresponding to the height in the depth image acquisition program. After performing the same processing on all eight calibrated heights, the regression equation for the relationship between grayscale value and height is obtained as follows: y=-5.2881x+212.31 The precision of the regression equation is R 2 =0.9972 Among them, x represents the height value and y represents the grayscale value.
2. The method for detecting uniformity of Flammulina velutipes based on a depth camera according to claim 1, characterized in that: The calculation method of the uniformity in step S4 is as follows: That is, the number of strains whose stipe heights in the whole cluster of Flammulina velutipes were within the range of ±5% of the average height was compared with the total number of stipes in the whole cluster of Flammulina velutipes, and the comparison results were presented in percentage.
3. A device for detecting uniformity of enoki mushrooms based on a depth camera, comprising a computer for executing the method for detecting uniformity of enoki mushrooms based on a depth camera according to claim 1 or 2, characterized in that: A Flammulina velutipes conveying device is provided, which includes a frame, a belt conveying device for conveying Flammulina velutipes mushroom clusters in a horizontal direction is provided on the top of the frame, and the Flammulina velutipes mushroom clusters are placed on the belt conveying device in a vertical position; a motor for driving the belt conveying device and a PLC control unit for controlling the start and stop of the motor are installed on the frame; An image acquisition device is provided, and a collection box is provided which is installed on the upper end of the frame and crosses the belt conveyor device. The collection box has a front door and a rear door for the belt conveyor device to pass through in sequence. A depth camera is arranged on the top wall inside the collection box and is centered above the belt conveyor device. The depth camera takes pictures of the enoki mushroom clusters on the belt conveyor device to obtain a depth image. There is also an infrared photoelectric sensor installed on the side of the belt conveyor device to detect whether there are Flammulina velutipes clusters and whether the Flammulina velutipes clusters are transported directly under the depth camera in a direction perpendicular to the conveying direction; An image storage device is also provided, which is connected to the depth camera and the computer respectively to store the depth image obtained by the depth camera, and transmit the depth image to the computer for uniformity detection of the enoki mushrooms; Among them, the motor, depth camera, and infrared photoelectric sensor are electrically connected to the PLC control unit to realize PLC automatic control.
4. The device for detecting uniformity of Flammulina velutipes based on a depth camera as claimed in claim 3, characterized in that: The top wall and side walls of the collection box are equipped with LED light sources for illuminating the interior of the collection box.
5. The device for detecting uniformity of Flammulina velutipes based on a depth camera as claimed in claim 3, characterized in that: The collection box is a square box, and the LED light source includes stepless dimming LED light strips arranged along the four vertical edges of the inner wall of the collection box, and the LED light source also includes stepless dimming LED light strips arranged along the four horizontal edges of the top wall of the collection box.
6. The device for detecting uniformity of Flammulina velutipes based on a depth camera as claimed in claim 5, characterized in that: The stepless dimming LED light strips at the four vertical edges of the collection box have their luminous surfaces facing the center line of the depth camera lens; the stepless dimming LED light strips at the four horizontal edges of the collection box have their luminous surfaces facing the intersection of the center line of the depth camera lens and the top surface of the belt conveyor device.
7. The device for detecting uniformity of Flammulina velutipes based on a depth camera as claimed in claim 3, characterized in that: The depth camera uses the second-generation Kinect camera released by Microsoft.
8. The device for detecting uniformity of Flammulina velutipes based on a depth camera as claimed in claim 3, characterized in that: The infrared photoelectric sensor is located directly opposite the shooting range of the depth camera.
9. The device for detecting uniformity of Flammulina velutipes based on a depth camera according to claim 3 or 8, characterized in that: The infrared photoelectric sensor is a diffuse reflection infrared photoelectric switch with model number E18-D80NK.
