Mushroom crack detection grading system based on yolov4

Abstract

A kind of mushroom crack detection grading system based on YOLOv4, including data acquisition module, data preprocessing module, data training module, grading module;The grading module is the application software installed in PC;The process of mushroom crack detection grading includes five steps.The application based on artificial intelligence technology, under the joint action of each unit, the end user will need to be graded mushroom under the camera of PC installed with corresponding application unit, after the camera collects data, the grading module can automatically grade the mushroom, and the detected mushroom grade can be obtained specifically, since artificial visual inspection is no longer needed, the grading cost and damage to mushroom are reduced, and the efficiency and grading quality of mushroom grading are improved, the labor cost is saved, which effectively helps the healthy development of mushroom industry.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of AI application, and particularly relates to a mushroom crack detection grading system based on YOLOv4. BACKGROUND

[0002] Mushrooms are nutritious foods with high protein and low fat, and have the characteristics of delicious taste and unique flavor. With the emphasis on health, more and more people begin to prefer to eat mushrooms, which makes the planting area and yield of mushrooms increase day by day. Because the quality grade of mushrooms is closely related to the nutritional components, mushrooms of different quality grades have different sales prices, and accurate and efficient grading of mushrooms can bring profit improvement to the growers and maintain the healthy development of the mushroom industry. According to the relevant industry standards, the quality grade of mushrooms is mainly related to the cap integrity, and the traditional grading method mainly relies on manual judgment of the number of cracks on the surface of mushrooms and the size of the cracks. This method obviously has the problems of low work efficiency, increased production cost, and non-uniform standards affected by subjective factors. At the same time, the manual grading of mushrooms is often affected by light, and different light intensities will cause the deviation between the image color and the true color, so that the quality of grading cannot be guaranteed, and the proportion of damage during grading is increased and the sales price of mushrooms is reduced in the artificial grading.

[0003] With the development of science and technology, artificial intelligence technology is more and more applied to the industrial field for quality identification of industrial products, but there is no mushroom grading system based on artificial intelligence in the prior art. Therefore, it is particularly necessary to provide a system based on artificial intelligence technology which can realize automatic grading of mushrooms. SUMMARY

[0004] In order to overcome the drawbacks of the existing artificial method for grading mushrooms as described in the background, the present application provides a mushroom crack detection grading system based on YOLOv4, which can automatically grade mushrooms under the joint action of each unit, reduces the grading cost and damage to mushrooms, and has higher grading efficiency and more accurate grading quality, thereby effectively assisting the healthy development of the mushroom industry.

[0005] The technical scheme adopted by the present application to solve the technical problems is:

[0006] A mushroom crack detection grading system based on YOLOv4, characterized by comprising a data acquisition module, a data preprocessing module, a data training module, and a grading module; the grading module is an application software installed in a PC; the process of mushroom crack detection grading is as follows, step (1): through the data acquisition module, different position and shape of mushroom images obtained by the camera are collected, and 70% of the collected data set is used as training data for the training model algorithm, and the remaining 30% is used as verification and test model algorithm detection effect, the training set image is copied into A, B, C, D four identical image sets by the data preprocessing module, and the test set is not copied. Then rotate, flip and crop each picture of A image set, and perform brightness transformation, contrast transformation and color transformation on each picture of B image set. Step (2): through the data preprocessing module, MG_Crack transformation is performed on C image set, and then the transformed A, B, C data set and the untransformed D image set are mixed to form a new training data set; the specific steps of MG_Crack image transformation are as follows: first, select the pictures with real box area greater than 0.3 and real box number greater than or equal to 3, and randomly select three pictures from these pictures without replacement, and randomly select one picture from the original mushroom crack data set without replacement, a total of four pictures, then make a circle with the diagonal intersection point as the center, randomly generate a segmentation point, and divide the three pictures into four parts, select the part with the largest average real box area in the four parts of one picture, select three parts from the three pictures, and finally splice the four corresponding parts of the fourth picture to form a picture for standby; step (3): through the data training module, the new training data set obtained in step (2) is input into the backbone network CSPDarknet53, then the extracted mushroom features are input into the PAN and SPP networks for feature fusion, and then the features are input into the YOLOv3 network for mushroom crack detection and recognition; step (4) through the data test module, the test set is input into the model, and finally the corresponding parameters are adjusted according to the recognition effect of the model on the test set to achieve the best performance, and after achieving the best performance, it can be used by the user; step (5): through the grading module, the terminal user collects the mushroom image data that needs to be graded through the camera and inputs it into the CSPDarknet53 network for feature extraction, then inputs the extracted features into the PAN and SPP networks for feature fusion, inputs the fused features into the YOLOv3 network for recognition, inputs the recognized mushroom crack coordinate information into the formula, obtains the quantitative grading information of each mushroom, and arranges all the mushroom grading values from small to large to divide the grades, so that the terminal user can get the detected mushroom quality.

[0007] Further, in the step (1), the rotation angle of each picture of the A image set is 90 degrees, 180 degrees and 270 degrees, the flipping mode is up-down flipping and left-right flipping, the cropping function is to expand the image data and enhance the robustness of the model detection; the brightness, contrast and color transformation of each picture of the B image set is to expand the image data and also enhance the robustness of the model detection; further, in the step (2), the MG_Crack transformation of the C image set is to enhance the recognition ability of the model algorithm to different scales of mushroom cracks; and the new training data set is to expand the training data set and improve the robustness and generalization ability of the algorithm model.

[0008] Further, in the step (3), the new training data set is input into the backbone network to better and more fully extract the features of the mushroom crack pictures, thereby improving the accuracy of the algorithm model; the extracted features are input into the PAN and SPP networks for feature fusion to further improve the performance of the algorithm model; and the features are input into the YOLOv3 network for mushroom crack detection and recognition to accurately recognize the features after extraction and fusion.

[0009] Further, in the step (2), the average value R of the mushroom [real box width average value + real box height average value] is taken as the radius to make a circle with the intersection point of the picture diagonal as the center, and then the picture is divided into four parts by randomly generating a segmentation point in the circle.

[0010] Further, in the step (1), the training set needs to be rotated by 90 degrees, 180 degrees and 270 degrees, and flipped up and down, flipped left and right, and cropped in the data augmentation process.

[0011] Further, in the step (5), the model algorithm is embedded into the APP of the Internet device in the actual process, and the embedded is the trained model, which can be directly used by the user for grading.

[0012] Further, in the step (5), in the grading of all mushrooms, the user can divide the grades according to his own needs.

[0013] The present application has the advantages that: based on artificial intelligence technology, the end user can place the mushrooms to be graded under the camera of the PC or the like installed with the corresponding application unit, the camera collects data, and the grading module can automatically grade the mushrooms, and the grade of the detected mushrooms can be specifically obtained, since the manual visual detection is no longer needed, the grading cost and the damage to the mushrooms are reduced, the efficiency and the quality of the mushroom grading are improved, the labor cost is saved, and the effective assistance is provided for the healthy development of the mushroom industry. BRIEF DESCRIPTION OF DRAWINGS

[0014] The accompanying drawings are included to provide a further understanding of the application, and are incorporated in and constitute a part of this specification, illustrate embodiments of the application, and are intended to explain the principles of the application, and are not intended to limit the application to the specific embodiments shown. In the drawings:

[0015] Figure 1 is a software architecture diagram of the present application.

[0016] Figure 2 is a radius determination diagram of the present application.

[0017] Figure 3 is a manual annotation diagram of the present application.

[0018] Figure 4 is a center determination diagram of the present application.

[0019] Figure 5 is a four-part division diagram of the present application.

[0020] Figure 6 is an invention development flow diagram.

[0021] Figure 7 is a flowchart of the MG_Crack data augmentation algorithm of the present application. DETAILED DESCRIPTION

[0022] Figure 1 , 6 As shown in the figure, a mushroom crack detection grading system based on YOLOv4 includes a data acquisition module, a data preprocessing module, a data training module, and a grading module; the grading module is an application software installed in a PC.

[0023] Figure 1 , 6The process of mushroom crack detection grading is shown in FIG. 7, as follows: (1) The data acquisition module collects images of mushrooms in different positions, different positions, and different shapes obtained by the camera. The data set collected is preprocessed by the data preprocessing module. 70% (2100 images) of the data set obtained is used as training data for the training model algorithm, and the remaining 30% (900 images) is used to verify and test the effect of the model algorithm recognition. Specifically, the training set needs to be duplicated into 4 parts, and the test set does not need to be duplicated. The training set images are duplicated into A, B, C, and D four identical image sets by the data preprocessing module, and then each image in the A image set is rotated (specifically including 90 degrees, 180 degrees, 270 degrees), flipped (specifically including up-down flipping and left-right flipping), and cropped (the functions of rotation, flipping, and cropping are to expand the image data, so that the image data is closer to the state of the mushroom in the real scene, and the robustness of the model detection is enhanced; since rotation, flipping, and cropping simulate the different changes of the same mushroom in the picture caused by different shooting angles and shooting distances in the real environment, the addition of these pictures after rotation, flipping, and cropping in the training set enables the algorithm model to fully learn the shape of the mushroom under different shooting angles and shooting distances, and thus better identify the different shapes of the mushroom caused by different shooting angles and shooting distances in the real scene); then, each image in the B image set is subjected to brightness transformation, contrast transformation, and color transformation (the functions of the above three transformation methods are to expand the image data, so that the image data is closer to the state of the mushroom in the real scene, and the robustness of the model detection is enhanced; because brightness transformation, contrast transformation, and color transformation simulate the different states of the same mushroom in the picture under different natural light, indoor light source, and different brightness, the addition of the pictures after the three changes in the training set enables the algorithm model to fully learn the shape of the mushroom under different light intensities and different light, and thus better identify the different shapes of the mushroom caused by different light intensities and different light in the real scene); (2) The data set is input into the model algorithm for training, and the model algorithm is trained to obtain the mushroom crack detection grading model; (3) The mushroom crack detection grading model is tested by using the test set, and the test result is obtained; (4) The test result is compared with the actual result, and the accuracy of the mushroom crack detection grading model is calculated; (5) The mushroom crack detection grading model is adjusted according to the accuracy of the mushroom crack detection grading model, and the process of steps (1) to (5) is repeated until the accuracy of the mushroom crack detection grading model is greater than or equal to 90%.

[0024] Figure 1 、 6As shown, the process of mushroom crack detection grading, step (2): through the data preprocessing module, the C image set is transformed by MG_Crack (data enhancement) (the role is to solve the problem of uneven or balanced distribution of mushroom crack area size in the training set, so that the number of mushroom cracks of different sizes is balanced, and then the ability of the model algorithm to identify mushroom cracks of different sizes is enhanced), then the transformed A, B, C image set and the untransformed D image set are mixed to form a new training data set (the role is to expand the training data set and improve the robustness and generalization ability of the algorithm model). It should be noted that the above operations are all completed on the training set, and the test set is not used from the beginning to the end in the whole process. The test set is used to evaluate the performance of the final model. The steps of MG_Crack image transformation are as follows: first, select the pictures with real frame area greater than 0.3 (pixel unit) and real frame (manually annotated frame) number greater than or equal to 3 (three cracks of mushrooms) from the training set, and randomly select three pictures from these pictures without replacement (for example, a total of 10 pictures are selected, the first time three pictures are randomly selected, then the three pictures are removed from the 10 pictures, three pictures are randomly selected from the remaining 7 pictures, and the process continues until there are less than three pictures left, then stop extracting), randomly select one picture from the original mushroom crack data set without replacement, take the average of ([real frame width average + real frame height average]) R as the radius of the circle with the intersection point of the picture diagonal line as the center, and then randomly generate a division point in the circle to divide the picture into four parts. Select the part with the largest average real frame area (Area i i i i i ) from the four parts. The above operation is for the three selected pictures, and the randomly selected picture in the original mushroom crack data set is only divided without selection (no Area i ). Finally, the three Area i (i = 1, 2, 3) of the three pictures are selected and combined with the corresponding part of the fourth picture, for example, the three Area i of the three pictures are A, C, and D, then the B part of the fourth picture is spliced with the three parts to form a picture. Because the division points of the four pictures are the same (the coordinates of the randomly generated points in the circle are the same in the four pictures), it is convenient to combine them.

[0025] Figure 1 、 6 ​​​​As shown in the process of mushroom crack detection grading, (3): the new training data set obtained in step (2) is input into the backbone network CSPDarknet53 network through the data training module (the role is to better and more fully extract the features of the mushroom crack picture, and then improve the accuracy of the algorithm model), and then the extracted features are input into the PAN and SPP networks for feature fusion (the role is to further extract the features of the mushroom crack picture, and fully fuse the features of different scales, enhance the semantic information of different feature layers, and further improve the performance of the algorithm model), and then the features are input into the YOLOv3 network for mushroom crack recognition (the role is to accurately recognize the extracted and fused features), and the mushrooms are graded according to the number of cracks and the size of the cracks of each mushroom picture.

[0026] Figure 1 、 6 As shown in the process of mushroom crack detection grading, step (4): the test set is input into the model through the data test module, and finally the corresponding parameters are adjusted to achieve the best performance according to the recognition effect of the model on the test set; after achieving the best performance, the model can be used by the user; in the actual process, the model algorithm is embedded into the APP of the Internet device, and the embedded is the trained model, which can be directly used by the user for grading.

[0027] Figure 1 、 6 As shown in the process of mushroom crack detection grading, step (5): through the grading module, the terminal user collects the mushroom image data that needs to be graded through the camera, and inputs the data into the CSPDarknet53 network for feature extraction, inputs the extracted features into the PAN and SPP networks for feature fusion, inputs the fused features into the YOLOv3 network for recognition, inputs the coordinate information of the recognized mushroom crack into the formula, obtains the quantitative grading information of each mushroom, and arranges all the grading values of the mushrooms from small to large, so that the user can obtain the tested mushroom quality; specifically, the user can divide according to his own needs, for example, if A, B and C are needed, the values arranged from small to large are divided into three equal parts.

[0028] Figure 1 As shown in the process of mushroom crack detection grading, step (2), the formula for determining the radius is, Where R represents the radius, such as Figure 2 The middle circle in the middle; And The three extracted pictures, wherein, W i represents the width of the mushroom crack bounding box, h jThe height of the bounding box representing the mushroom crack is typically manually annotated after image acquisition before it can be used for model training. Figure 3 The rectangle represents the bounding box, where the horizontal axis represents the width and the vertical axis represents the height. 'n' represents the total number of mushroom cracks in the three randomly selected images. The main purpose of this formula is to use the average width and height of the mushroom crack bounding boxes in the three images as the radius of the circle. This helps to better determine the range of the segmentation points, ensuring that the four segments are neither too large nor too small. Since the center of the circle is generally a fixed value, and the input dataset is determined, the center is also determined. The center is labeled (X, Y), where... like Figure 4 The middle point is the center of the circle (X, Y). W Circle, H 圆 These represent the width and height of the input image, respectively. Figure 4 In the diagram, the horizontal axis represents the width, and the vertical axis represents the height. Specifically, x = Random(XR, X+R) and y = Random(YR, Y+R). Note: [The last part is incomplete and likely refers to a different context.] Figure 4 The top left corner is the center of the circle, the horizontal axis is the positive half-axis of the x-axis, and the vertical axis is the positive half-axis of the y-axis. Most algorithms establish the coordinate system in this way. In step (2), a point (x, y) is randomly established inside the circle. The random points established in the four extracted images are the same, which facilitates the subsequent stitching. Randomly finding a point inside the circle, rather than selectively finding a point, is mainly to introduce randomness and prevent interference from subjective human factors. In step (2), Area i =max{areaA i areaB i areaC i areaD i}i = 1, 2, 3, specifically, select the part with the largest average crack area in each of the 3 images, where areaA i areaB i areaC i area D i represents the four segments of an image, and i indicates which image it is. For example, if i = 1, areaA1 represents segment A of the first image. (X = A, B, C, D, i = 1, 2, 3, j = 1, 2, ..., n), where n represents the number of cracks, and areaX i This represents the average area of ​​the crack in each part of each image, e.g., X = A, i = 1, j = 3, areaA 13 This is represented as the third mushroom crack in section A of the first image, such as... Figure 5 The diagram shows four parts: A, B, C, and D. The method for calculating the area of ​​a single mushroom-shaped crack is as follows:

[0029] areaX ij = (width*height) / 2 the area of the bounding box of the mushroom crack is approximated by (width*height) / 2, wherein width is the width of the bounding box, height is the height of the bounding box, and the part with the largest average area of the mushroom crack is selected (three pictures are randomly selected, and the part with the largest average area is selected from each picture, and three parts with the largest average area are selected, and the corresponding part of the fourth picture is spliced) mainly to enrich the background of the original image (more mushroom images and less background) and balance the number of small cracks and large cracks in the mushroom (according to statistical analysis, the number of small cracks is often more, and the number of large cracks is relatively less, which is not conducive to the identification of large cracks by the model, so as to improve the performance of the model).

[0030] Figure 1 6 As shown in the formula, in step (5), the evaluation formula of the mushroom grading is as follows, Wherein, λ is a weight factor, and n is the number of mushroom cracks in a picture, the formula well balances the influence of crack area and crack number on the evaluation of mushroom grading, and makes the grading of the mushroom have a quantitative description. The selection of λ: (several parameter values of λ are obtained through experiments), specifically, the influence of crack area less than 0.01 on the mushroom grading is small, so λ=1, between 0.01 and 0.1, because the area is small, even if there are many cracks in a mushroom within this range, the value added by the small area may not be greater than that of one or two or three relatively large cracks, but the influence on the grading is greater than that of one or two or three relatively large cracks, so the weight factor is appropriately increased, and the others are the same, when the area is relatively large, the value of λ is reduced, and the influence of the crack area on the grading is reduced, which is specifically expressed by the following formula,

[0031] if area≤0.01, λ=1;

[0032] 0.01<area<0.1, λ=5;

[0033] 0.1≤area<0.20, λ=2;

[0034] 0.2≤area<0.3, λ=1;

[0035] 0.3≤area<0.5, λ=0.8;

[0036] area≥0.5, λ=0.5.

[0037] ​​Through the above, based on artificial intelligence technology, the application will need to be classified mushrooms in the application terminal user is placed under the camera of PC and the like installed with the corresponding application unit, the camera collects data, through the grading module can automatically classify the mushrooms, can specifically obtain the detected mushroom grade, since no longer need human visual detection, reduces the grading cost and the damage of mushrooms, and improves the efficiency and grading quality of the mushroom grading, saves the labor cost, and effectively helps the healthy development of the mushroom industry.

[0038] It should be noted that although the embodiments of the present application have been shown and described, those skilled in the art can understand that various changes, modifications, replacements and variations can be made to these embodiments without departing from the principles and spirits of the present application, and therefore the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A YOLOv4-based mushroom crack detection grading system, characterized in that, Including data acquisition module, data preprocessing module, data training module, hierarchical module;The hierarchical module is the application software installed in the PC;The process of mushroom crack detection grading is as follows, step (1): through the data acquisition module, the mushroom images of different positions and different shapes obtained by the camera are collected, and 70% of the collected data set is used as the training data of the training model algorithm through the data preprocessing module, and the remaining 30% is used as the verification and test model algorithm detection effect, the training set image is copied into A, B, C, D four identical image sets through the data preprocessing module, the test set is not copied;Then rotate, flip and crop each picture of A image set, respectively, and perform brightness transformation, contrast transformation and color transformation on each picture of B image set;Step (2): through the data preprocessing module, MG_Crack transformation is performed on C image set, and then A, B and C image sets after transformation and D image set without transformation are mixed to form a new training data set;The specific steps of MG_Crack image transformation are as follows, first, select the pictures with real box area greater than 0.3 and real box number greater than or equal to 3, and randomly select three pictures from these pictures without replacement, and randomly select one picture from the original mushroom crack data set without replacement, a total of four pictures, then make a circle with the diagonal intersection point as the center, randomly generate a segmentation point, and divide the three pictures into four parts, and select the part with the largest average real box area in the four parts of one picture, three parts are selected from three pictures, and finally the four corresponding parts of the fourth picture are spliced into one picture for standby;Step (3): through the data training module, the new training data set obtained in step (2) is input into the backbone network CSPDarknet53, then the extracted mushroom features are input into the PAN and SPP networks for feature fusion, and then the features are input into the YOLOv3 network for mushroom crack detection and recognition;Step (4) through the data test module, the test set is input into the model, and finally the corresponding parameters are adjusted according to the recognition effect of the model on the test set to achieve the best performance, and after achieving the best performance, it can be used by the user;Step (5): through the hierarchical module, the terminal user collects the mushroom image data that needs to be graded through the camera, and inputs it into the CSPDarknet53 network for feature extraction, then inputs the extracted features into the PAN and SPP networks for feature fusion, inputs the fused features into the YOLOv3 network for recognition, inputs the recognized mushroom crack coordinate information into the formula, obtains the quantitative grading information of each mushroom, and arranges all the mushroom grading values from small to large to divide the grades, so that the terminal user can get the detected mushroom quality.

2. The YOLOv4-based mushroom crack detection and grading system according to claim 1, wherein, In step (1), the rotation angle of each picture of the A image set is 90 degrees, 180 degrees, and 270 degrees, the flipping mode is up-down flipping and left-right flipping, and the cropping function is to expand image data and enhance the robustness of the model detection; the brightness, contrast, and color transformation of each picture of the B image set are to expand image data and enhance the robustness of the model detection; further, in step (2), the MG_Crack transformation of the C image set is to enhance the recognition ability of the model algorithm for different scales of mushroom cracks; The function of constituting a new training data set is to expand the training data set and improve the robustness and generalization ability of the algorithm model.

3. The YOLOv4-based mushroom crack detection and grading system of claim 1, wherein, In step (3), inputting the new training data set into the backbone network is to better and more fully extract the features of the mushroom crack pictures, thereby improving the accuracy of the algorithm model; inputting the extracted features into the PAN and SPP networks for feature fusion is to further improve the performance of the algorithm model; and inputting the features into the YOLOv3 network for mushroom crack detection and recognition is to accurately recognize the extracted and fused features.

4. The YOLOv4-based mushroom crack detection and grading system of claim 1, wherein, In step (2), the average value of the average value of the real box width and the real box height is taken as the radius R to make a circle with the intersection point of the picture diagonal as the center, and then the picture is divided into four parts by randomly generating a segmentation point in the circle.

5. The YOLOv4-based mushroom crack detection and grading system of claim 1, wherein, In step (1), the training set needs to be rotated by 90 degrees, 180 degrees, and 270 degrees, and flipped up and down, left and right, and cropped, and brightness, contrast, and color transformed during the data augmentation process.

6. The YOLOv4-based mushroom crack detection and grading system of claim 1, wherein, In step (5), the model algorithm will be embedded into the APP of the internet device in the actual process, and the embedded is the trained model, which can be directly used for grading by the user.

7. The YOLOv4-based mushroom crack detection and grading system of claim 1, wherein, In step (5), in the grading of all mushrooms, the user can divide the grades according to his own needs.

Images