Corn seed direction identification method based on multi-feature fusion

Through the improved FCOS object detection algorithm and multi-feature fusion technology, combined with data enhancement and angle correction, the problem of poor artificial intervention and generalization ability in corn seed direction recognition is solved, and the accuracy and rapid identification of corn seed direction is achieved. in the actual production environment.

CN119006774APending Publication Date: 2024-11-22HARBIN INST OF TECH
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202410947568.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-16
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The existing corn seed direction recognition methods have strong manual intervention and poor generalization ability, making it difficult to achieve accurate, fast and low-cost corn seed direction recognition in actual production, affecting the corn breeding process.

Method used

Using an improved FCOS object detection algorithm based on multi-feature fusion, through data augmentation and angle correction, combined with industrial camera acquisition images, rolabelImg annotation and data division, the model is trained to achieve automatic identification of corn seed direction.

Benefits of technology

It realizes accurate and rapid identification of corn seed direction, reduces identification costs, is applicable to the actual production environment, and improves the automation and efficiency of the breeding process.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119006774A_ABST
    Figure CN119006774A_ABST
Patent Text Reader

Abstract

The invention provides a corn seed direction identification method based on multi-feature fusion, and belongs to the technical field of seed direction identification. The method comprises the following steps: S1, acquiring a corn seed image; step S2, marking the corn seed image by adopting rolaabelImg; s3, performing data enhancement on the obtained labeled data set, and dividing the enhanced data set into a training set, a verification set and a test set according to a proportion of 8: 1: 1; and S4, an improved FCOS target detection algorithm is selected, the optimal weight and parameters of a target detection algorithm model are obtained through training based on the training set divided in the step S3, after verification of the verification set, testing is conducted through the test set, and when mAP indexes on the test set meet requirements, the trained improved FCOS target detection algorithm can be used for orientation recognition of corn seeds. The multi-feature fusion corn seed direction identification method provided by the invention is simple and easy to implement, strong in generalization ability and capable of automatically learning hierarchical features, and the method is low in cost, high in precision, high in identification speed and suitable for being applied to actual production.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of seed direction recognition, and in particular to a corn seed direction recognition method based on multi-feature fusion. Background Art

[0002] As an important industrial raw material and feed crop, corn occupies an important position in my country's economic development. However, my country's corn seed industry faces difficulties such as homogeneity of varieties and lack of special corn varieties. Molecular marker-assisted breeding can reduce the scale of population planting for new variety cultivation. Molecular marker-assisted breeding automation can be realized through the automatic slicing equipment for corn seeds, which greatly accelerates the process of corn breeding. The embryo is extremely important for corn seeds and is extremely sensitive to any damage. Its integrity and vitality are crucial for the successful germination of corn seeds and plant growth. Therefore, before the automatic laser slicing of corn seeds, it is necessary to accurately identify the direction of the corn seeds and adjust the posture through mechanical grippers to avoid damaging the embryo and reducing seed activity during laser slicing. Corn seeds have irregular appearance characteristics, large differences between varieties, and common deformation of seed shape. In addition, the direction of single corn seeds on the feeding tray is randomly distributed, and it is difficult to achieve precise orientation only through mechanical devices. At present, there are some methods for identifying the direction of corn seeds, such as using grayscale histogram segmentation thresholds to obtain binary images and compare areas, setting thresholds based on grayscale median filter denoised images to extract perimeter, area, and circularity parameter features, and using neural network pattern recognition toolboxes to build models, etc. However, these methods have defects such as strong manual intervention and poor generalization ability, and are not suitable for actual production. In actual production, recognition accuracy, generalization ability, and recognition cost are all important factors affecting the breeding process. Therefore, how to accurately, generalize, and low-cost identify the direction of corn seeds is of great significance for realizing automated molecular marker-assisted breeding. Summary of the invention

[0003] The main purpose of the present invention is to provide a corn seed direction recognition method based on multi-feature fusion, which can effectively solve the problems in the background technology.

[0004] To achieve the above object, the technical solution adopted by the present invention is:

[0005] A corn seed direction recognition method based on multi-feature fusion, the recognition method comprising the following steps:

[0006] Step S1: collecting corn seed images;

[0007] Step S2: using rolabelImg to label the corn seed image collected in step S1;

[0008] Step S3: perform data augmentation on the labeled data set obtained in step S2, and divide the augmented data set into a training set, a validation set, and a test set according to a ratio of 8:1:1;

[0009] Step S4: Select the improved FCOS target detection algorithm based on the training set divided in step S3, train to obtain the optimal weights and parameters of the target detection algorithm model, verify it with the validation set, and then test it with the test set. If the mAP index meets the requirements on the test set, the trained improved FCOS target detection algorithm can be used for corn seed orientation recognition.

[0010] Preferably, in step S1, an industrial camera is used to capture the corn seed image. When capturing the corn seed image, the illumination of the corn seed and the background are kept consistent, and the background is selected to be black with strong light absorption.

[0011] Preferably, in step S2, the rotating rectangular frame is the circumscribed rectangle of the seed during marking, and the aspect ratio of the rectangular frame is reasonably set to avoid the appearance of a square. During manual marking, the tip of the corn seed, i.e., the front end, is on the perpendicular bisector of one of the sides, and the blunt end, i.e., the rear end, is tangent to the other side, thereby ensuring that the direction of the corn seed embryo is the same as the direction of one of the two vertical sides of the rotating rectangular frame.

[0012] Preferably, the data enhancement described in step S3 includes blurring, flipping, rotating, saturation adjustment and perspective transformation.

[0013] Preferably, the improved FCOS target detection algorithm model in step S4 adds a channel convolution layer to the head regression branch of the FCOS network, the output channel of the convolution layer is 1, the convolution kernel size is 3×3, and the rotation angle information of the algorithm-predicted target box is regressed, combined with the bounding box position information predicted by the model, to achieve rotated target detection, and give the model-predicted corn seed rotation angle value.

[0014] Preferably, the orientation recognition in step S4 is achieved by controlling the intersection-over-union ratio (KF-IOU) and angle correction between the rotated rectangular box annotated in the training set and the target box predicted by regression of the improved FCOS target detection algorithm model.

[0015] Preferably, the KF-IOU is an improvement of the regression loss function IoU loss in the FCOS network.

[0016] Preferably, the angle correction is based on the improved FCOS target detection algorithm model, the center point and four vertex coordinates of the regression prediction target frame, the prediction area is divided into four triangular areas by two adjacent vertices and the center point, the HSV color channel of each area is separated, the S channel mean of each area is calculated for comparison, and the triangular area with the smallest S channel mean is taken; the coordinates of the two vertices constituting the triangular area are extracted, the midpoint of the two vertices is calculated, and the direction from the midpoint to the center point of the regression prediction target frame is determined as the identification direction of the embryo tip of the corn seed. The specific direction of the corn seed is represented by the angle with the vertical line passing through the center point, and the range is [-180°, 180°], clockwise is positive, and counterclockwise is negative.

[0017] Compared with the prior art, the present invention has the following beneficial effects:

[0018] The corn seed direction recognition method of the present invention is different from the existing seed direction recognition method. The present invention comprehensively considers the morphological characteristics and color characteristics of corn seeds, improves the FCOS target detection algorithm, and proposes a simple, easy, generalized, and multi-feature fusion corn seed direction recognition method that can automatically learn hierarchical features. This method has low cost, high accuracy, and fast recognition speed, and is suitable for use in actual production. The direction of corn seeds at the feeding tray of the corn seed automatic slicer is recognized by the recognition method of the present invention, which effectively guarantees the activity of the corn seeds after slicing. The present invention helps to realize the integration and automation of corn seed slicing, has strong applicability, and is suitable for universal promotion. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a schematic diagram of the four-interval division of the prediction area in Example 2;

[0020] Figure 2 The prediction block diagram of direction identification of corn seeds with obvious direction differences in some large-angle distribution ranges in Example 1 in Example 2 and the prediction block diagram after angle correction;

[0021] Figure 3 The prediction block diagram of direction recognition of corn seeds with obvious morphological differences and the prediction block diagram after angle correction in Example 2 of Example 2;

[0022] Figure 4 This is the prediction block diagram for identifying the direction of corn seeds with opposite embryo tip directions in Example 3 in Example 2 and the prediction block diagram after angle correction. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] Example 1

[0025] A corn seed direction recognition method based on multi-feature fusion, the recognition method comprising the following steps:

[0026] Step S1: collecting corn seed images;

[0027] Step S2: using rolabelImg to label the corn seed image collected in step S1;

[0028] Step S3: perform data augmentation on the labeled data set obtained in step S2, and divide the augmented data set into a training set, a validation set, and a test set according to a ratio of 8:1:1;

[0029] Step S4: Select the improved FCOS target detection algorithm based on the training set divided in step S3, train to obtain the optimal weights and parameters of the target detection algorithm model, verify it with the validation set, and then test it with the test set. If the mAP index meets the requirements on the test set, the trained improved FCOS target detection algorithm can be used for corn seed orientation recognition.

[0030] Preferably, in step S1, an industrial camera is used to capture the corn seed image. When capturing the corn seed image, the illumination of the corn seed and the background are kept consistent, and the background is selected to be black with strong light absorption.

[0031] Preferably, in step S2, the rotating rectangular frame is the circumscribed rectangle of the seed during marking, and the aspect ratio of the rectangular frame is reasonably set to avoid the appearance of a square. During manual marking, the tip of the corn seed, i.e., the front end, is on the perpendicular bisector of one of the sides, and the blunt end, i.e., the rear end, is tangent to the other side, thereby ensuring that the direction of the corn seed embryo is the same as the direction of one of the two vertical sides of the rotating rectangular frame.

[0032] Preferably, the data enhancement described in step S3 includes blurring, flipping, rotating, saturation adjustment and perspective transformation.

[0033] Preferably, the improved FCOS target detection algorithm model in step S4 adds a channel convolution layer to the head regression branch of the FCOS network, the output channel of the convolution layer is 1, the convolution kernel size is 3×3, and the rotation angle information of the algorithm-predicted target box is regressed, combined with the bounding box position information predicted by the model, to achieve rotated target detection, and give the model-predicted corn seed rotation angle value.

[0034] Preferably, the orientation recognition in step S4 is achieved by controlling the intersection-over-union ratio (KF-IOU) and angle correction between the rotated rectangular box annotated in the training set and the target box predicted by regression of the improved FCOS target detection algorithm model.

[0035] Preferably, the KF-IOU is an improvement of the regression loss function IoU loss in the FCOS network. It solves the problem that the gap between non-overlapping frames cannot be evaluated during the training process, speeds up the convergence speed, and can better guide the training process of the model.

[0036] Preferably, the angle correction is based on the improved FCOS target detection algorithm model, the center point and four vertex coordinates of the regression prediction target frame, the prediction area is divided into four triangular areas by two adjacent vertices and the center point, the HSV color channel of each area is separated, the S channel mean of each area is calculated for comparison, and the triangular area with the smallest S channel mean is taken; the coordinates of the two vertices constituting the triangular area are extracted, the midpoint of the two vertices is calculated, and the direction from the midpoint to the center point of the regression prediction target frame is determined as the identification direction of the embryo tip of the corn seed. The specific direction of the corn seed is represented by the angle with the vertical line passing through the center point, and the range is [-180°, 180°], clockwise is positive, and counterclockwise is negative.

[0037] Example 2

[0038] like Figure 1-4 As shown, a corn seed direction recognition method based on multi-feature fusion, the recognition method comprises the following steps:

[0039] Step S1: Collect corn seed images

[0040] The corn seed image can be collected by a digital camera or an industrial camera. In this embodiment, the corn seeds at the loading tray of the automatic seed slicer are preferably collected by an industrial camera. During the collection, the consistency of the lighting of the corn seeds and the background environment must be ensured, and the background is selected to be black with strong light absorption for easy subsequent processing.

[0041] Step S2: Image annotation

[0042] Each corn seed in the image collected in step S1 is labeled by rolabelImg. During labeling, the rotated rectangular box is the circumscribed rectangle of the seed. The regression effect of the high aspect ratio detection box is better. The aspect ratio of the rectangular box and the seed circumscribed rule are reasonably set. The two parallel sides of the rectangular box should be ensured to be perpendicular to the tip of the corn seed embryo and the embryo tip is located on its perpendicular bisector, and the other side is tangent to the dorsal side of the corn embryo tip. The direction of the seed embryo should be the long side of the rotated rectangular box as much as possible, so as to further improve the detection accuracy of the network model.

[0043] Step S3: Image data enhancement

[0044] The annotated data set obtained in step S2 is enhanced by data enhancement, including blurring, flipping, rotating, saturation adjustment, and perspective transformation, to obtain an enhanced data set, and the enhanced data set is divided into a training set, a validation set, and a test set in a ratio of 8:1:1. Data enhancement can increase the amount of data, prevent overfitting, and improve the robustness of the model; dividing the validation set and the test set can optimize the use of computing resources. By evaluating the model performance on independent validation sets and test sets, the generalization ability of the model can be evaluated, the model overfitting can be prevented, and the reliability of the model in practical applications can be improved.

[0045] Step S4: Based on the improved FCOS target detection algorithm model trained by the training set images divided by step S3, after verification by the verification set, the prediction set is tested to realize the orientation recognition of corn seeds. The orientation recognition is realized by controlling the intersection-over-union ratio KF-IoU and angle correction of the target box and the prediction box in the target detection algorithm:

[0046] The improved FCOS algorithm model is to add a channel convolution layer to the head regression branch of the FCOS network. The output channel of the convolution layer is 1, and the convolution kernel size is 3×3, which is used to regress the angle information of the rotation box, connect the 4-dimensional bounding box prediction information and the 1-dimensional angle prediction information, and finally realize the rotation box prediction;

[0047] The KF-IoU is an improvement of the regression loss function IoU loss in the FCOS network. It solves the problem that the gap between non-overlapping frames cannot be evaluated during the training process, speeds up the convergence speed, and can better guide the training process of the model.

[0048] The angle correction is based on the improved FCOS target detection algorithm model, regressing and predicting the center point and four vertex coordinates of the target frame, and dividing the prediction area into four triangular areas through two adjacent vertices and the center point. Figure 1 A schematic diagram of the area division;

[0049] Select a 6000×4000 pixel seed image, separate the HSV color channels of the image, extract the mean values ​​of the H, S, and V channels of the four regions respectively, and select the S channel with the highest discrimination in each region as the selection basis. The mean value of the S channel region 1 is 0.0244, the mean value of region 2 is 0.058, the mean value of region 3 is 0.101, and the mean value of region 4 is 0.037. Select the triangular region with the smallest mean value of the S channel;

[0050] Extracting the coordinates of two vertices constituting the triangular area, obtaining the midpoint of the two vertices, determining the corn seed embryo tip from the center point to the midpoint, and correcting the corn seed identification direction;

[0051] The specific direction of the corn seed is represented by the angle between the corn seed and the vertical line passing through the center point, ranging from [-180°, 180°], with clockwise being positive and counterclockwise being negative.

[0052] Example 1: As attached Figure 2 As shown, attached Figure 2 The following are the prediction diagrams for the direction identification of corn seeds with obvious direction differences in the large angle distribution range in the experiment and the prediction diagrams after angle correction. Figure 2 It can be seen that the direction of the corn seed embryo tip is randomly distributed at the loading tray, and the angular distribution range is very wide. The corn seed direction recognition method can realize the accurate detection of the direction of the corn seed embryo tip in a large angular distribution range, and has strong generalization.

[0053] Example 2: As attached Figure 3 As shown, attached Figure 3 The following are the prediction diagrams for the orientation recognition of some corn seeds with obvious morphological differences and the prediction diagrams after angle correction. Figure 3 It can be seen that corn seeds have grain deformation, and the deformation types are diverse and uncontrollable, resulting in obvious differences in the morphology of different corn seeds. The corn seed direction recognition method can accurately identify the directions of conventional corn seeds and various distorted corn seeds, and has strong generalization.

[0054] Example 3: As attached Figure 4 As shown, attached Figure 4 The prediction diagram for the direction identification of some corn seeds with opposite embryo tips in the experiment and the prediction diagram after angle correction are shown in the attached figure. Figure 4 It can be seen that after the introduction of angle correction, the orientation of the corn seed embryo tip can be accurately detected, and the detection of corn seeds in opposite directions can be achieved.

[0055] The corn seed direction recognition method based on morphological features of the present invention is used to carry out direction recognition tests on Northeast Africa 001, Jinongda 39 and Xianyu 335 corn seeds at the loading tray of an automatic seed slicer, and recognition tests are carried out on common corn seed varieties. The average recognition accuracy is 90.9%, the recognition time is better than 1s, and the germination rate of seeds after slicing is 98%, which fully meets the production needs. The method of the present invention is simple, has high accuracy, less calculations, high speed, low cost, strong practicability, and is suitable for actual production.

[0056] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. Those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some of the technical features therein with equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A corn seed direction recognition method based on multi-feature fusion, characterized in that: The identification method comprises the following steps: Step S1: collecting corn seed images; Step S2: using rolabelImg to label the corn seed image collected in step S1; Step S3: perform data augmentation on the labeled data set obtained in step S2, and divide the augmented data set into a training set, a validation set, and a test set according to a ratio of 8:1:1; Step S4: Select the improved FCOS target detection algorithm based on the training set divided in step S3, train to obtain the optimal weights and parameters of the target detection algorithm model, verify it with the validation set, and then test it with the test set. If the mAP index meets the requirements on the test set, the trained improved FCOS target detection algorithm can be used for corn seed orientation recognition.

2. The corn seed direction recognition method based on multi-feature fusion according to claim 1 is characterized in that: In the step S1, an industrial camera is used to collect corn seed images. When collecting the corn seed images, the illumination of the corn seeds and the background are kept consistent, and the background is selected to be black with strong light absorption.

3. The corn seed direction recognition method based on multi-feature fusion according to claim 1 is characterized in that: In step S2, the rotating rectangular frame is the circumscribed rectangle of the seed during marking, and the aspect ratio of the rectangular frame is reasonably set to avoid the appearance of a square. During manual marking, the tip of the corn seed, i.e., the front end, is on the perpendicular bisector of one of the sides, and the blunt end, i.e., the rear end, is tangent to the other side, so as to ensure that the direction of the corn seed embryo is the same as the direction of one of the two vertical sides of the rotating rectangular frame.

4. The corn seed direction recognition method based on multi-feature fusion according to claim 1, characterized in that: The data enhancement described in step S3 includes blurring, flipping, rotation, saturation adjustment, and perspective transformation.

5. The corn seed direction recognition method based on multi-feature fusion according to claim 1, characterized in that: In the step S4, the improved FCOS target detection algorithm model adds a channel convolution layer to the head regression branch of the FCOS network, the output channel of the convolution layer is 1, the convolution kernel size is 3×3, the rotation angle information of the algorithm-predicted target box is regressed, and the bounding box position information predicted by the model is combined to realize the rotation target detection, and the rotation angle value of the corn seed predicted by the model is given.

6. The corn seed direction recognition method based on multi-feature fusion according to claim 3 or 5, characterized in that: In step S4, orientation recognition is achieved by controlling the intersection-over-union (KF-IOU) of the rotated rectangular box annotated in the training set and the target box predicted by the regression of the improved FCOS target detection algorithm model and angle correction.

7. The corn seed direction recognition method based on multi-feature fusion according to claim 6 is characterized in that: The KF-IOU is an improvement of the regression loss function IoUloss in the FCOS network.

8. The corn seed direction recognition method based on multi-feature fusion according to claim 6, characterized in that: The angle correction is based on the improved FCOS target detection algorithm model, regresses the center point and four vertex coordinates of the target frame, divides the prediction area into four triangular areas through two adjacent vertices and the center point, separates the HSV color channel of each area, calculates the S channel mean of each area for comparison, and takes the triangular area with the smallest S channel mean; extracts the coordinates of the two vertices constituting the triangular area, calculates the midpoint of the two vertices, and determines the corn seed embryo tip identification direction from the midpoint to the center point of the regression prediction target frame. The specific direction of the corn seed is represented by the angle with the vertical line passing through the center point, and the range is [-180°, 180°], clockwise is positive, and counterclockwise is negative.

Citation Information

Patent Citations

  • Unmanned vehicle carried camera target detection method based on improved FCOS algorithm

    CN115761667A