A navel orange recognition method based on contour analysis and gridding

Abstract

The present invention discloses a navel orange recognition method based on contour analysis and gridding, and relates to the technical field of navel orange recognition. The method of the present invention combines contour analysis with gridding processing strategies, effectively solving the problems of illumination changes, fruit stacking and leaf occlusion faced by the prior art in navel orange recognition, while improving recognition speed and accuracy. The performance of experimental data on a standard navel orange image dataset shows that the method not only performs well in recognition accuracy, but also exhibits good adaptability to navel oranges of different sizes and shapes, opening up a new path for intelligent agricultural applications such as automated picking and grading, indicating that in the future, through continuous optimization of methods, the recognition efficiency and accuracy of navel oranges and other fruits and vegetables can be further improved.

Description

Technical Field

[0001] The invention relates to the technical field of navel orange recognition, and in particular to a navel orange recognition method based on contour analysis and gridding. Background Art

[0002] Researchers have developed a series of methods to improve the accuracy and efficiency of navel orange identification and provide strong technical support for applications in related fields. At present, there are mainly the following types of navel orange identification or detection methods:

[0003] 1) Single feature based method

[0004] Early harvesting robots mainly relied on black and white sensors or low dynamic range color cameras, and the information provided by the sensors was limited. Single-feature-based recognition methods mainly use the differences between shallow features such as color, texture, size, and shape to identify and detect fruits. For example, Hao Yong et al. used visible / near-infrared spectral analysis combined with soft independent pattern classification and partial least squares discriminant analysis pattern recognition methods to identify the varieties of navel oranges in southern Jiangxi. Lei Xiangming et al. proposed a citrus rapid recognition method based on the watershed method to address the problem of citrus adhesion and overlap and difficulty in identification in natural environments. In order to accurately identify citrus fruit diseases and pests and improve the level of informatization of citrus production, Li Yuan et al. proposed an optimal RGB linear combination color model for target recognition.

[0005] 2) Method based on multi-feature fusion

[0006] In recent years, with the development of hardware technology, the resolution of some color cameras has been improved, and they have the functions of anti-glare, automatic brightness compensation, high dynamic range, etc., which can obtain high-quality color images with rich information. Therefore, the multi-feature detection method has gradually replaced the single-feature detection method. Compared with the single-feature method, the core idea of ​​the multi-feature fusion method is to encode the pixel or the local features of the pixel to form a feature descriptor. This method uses similarity measurement and superpixel and regional fusion methods to screen potential navel orange areas and extract regional features as the input of the classifier to complete the classification of the region. Yu Changgeng et al. proposed a navel orange recognition method based on wavelet transform and Otsu threshold denoising. By establishing a YCbCr color model that is conducive to image segmentation, they proposed a centroid circle filling method to determine the position of the navel orange in the image. Chu Bowen et al. extracted the image features of oranges through image processing, and imported the extracted feature set into the BP neural network for training to complete classification and recognition.

[0007] 3) Deep learning-based methods

[0008] At present, recognition and detection methods based on convolutional neural networks have gradually been applied to fruit recognition and detection tasks. Compared with traditional detection methods, the feature information generated by the forward propagation process of convolutional neural networks exists in the form of feature maps, which retains the location information of the features. Methods based on deep learning can flexibly construct segmentation and detection frameworks based on feature maps of different resolutions, and integrate the two to target more complex tasks. For example, Ji et al. designed a multi-mode assisted YOLOv5 model to solve the problem that the illumination under outdoor sunlight affects the accuracy of fruit recognition. Xiong Juntao et al. improved the detection network based on the YOLOv3 method and identified citrus in a night environment. The average recognition accuracy on the test set was 90.75%, and the recognition speed was 53fps. Xiong Zhengwu et al. established a navel orange fruit recognition model that combines fast guided filtering and deep learning. The model has high robustness and real-time performance for navel orange fruit recognition in natural environments.

[0009] Existing navel orange recognition methods are mainly divided into three categories: single feature, multi-feature and deep learning. Although these methods can effectively use image information and advanced features to improve the accuracy and efficiency of navel orange recognition, they still cannot completely overcome the recognition difficulties caused by external interference factors such as changing lighting conditions, fruit stacking and leaf occlusion.

[0010] The recognition method based on single feature is limited by the limited feature information, and its accuracy and robustness need to be improved; although the recognition method integrating multiple features improves the recognition accuracy, the feature extraction process is complex and may lack representativeness; although the recognition method based on deep learning has strong adaptability to complex environments, problems such as long training time and high consumption of computing resources still need to be solved.

[0011] Therefore, a new solution to the above problems needs to be proposed. Summary of the invention

[0012] The object of the present invention is to provide a navel orange recognition method based on contour analysis and gridding to solve the technical problems raised in the background technology.

[0013] To achieve the above object, the present invention provides the following technical solution: a navel orange identification method based on contour analysis and gridding, comprising at least the following steps:

[0014] S1: For the navel orange image obtained by the picking robot, the orange pixel points are retained using the orange RGB range;

[0015] S2: Draw the contour on the original image based on the retained orange pixels;

[0016] S3: Obtain contour points by removing noise contours;

[0017] S4: Divide the image into several grids and record the number of initial contour points in each grid;

[0018] S5: Draw circles one by one using contour points and calculate their scores;

[0019] S6: Remove inaccurate circles: circles with low orange coverage, noise circles, and small overlapping circles;

[0020] S7: Draw an accurate circle on the original image to obtain an image containing one or more circles, and identify the position and size of the navel orange through the image.

[0021] Preferably, in S3, the area of ​​each contour is calculated, and the noise contour is removed according to the contour scaling factor;

[0022] The noise profile is defined as follows:

[0023]

[0024] Where S is the noise profile, S countour_max is the upper limit of the number of contours, It is assumed that the contour scale factor is the noise scale factor.

[0025] Preferably, the step 5 at least includes the following steps:

[0026] Step 51: randomly select 3 contour points from different grids to draw a new circle, and calculate its score, where the score is the accuracy of the circle. The new circle is two blue circles and one red circle, and the red circle is located between the blue circles.

[0027] Step 52: Delete the contour points on the new circle;

[0028] Step 53: Delete the meshes with less than 20% of the initial contour points;

[0029] Step 54: Traverse the grid and repeat steps 51 to 53 until the number of grids is less than 3. When the number of grids is less than 3, it is not enough to support drawing circles. After the loop ends, a set of circles and their scores are obtained.

[0030] Preferably, step 6 at least includes the following steps:

[0031] Step 61: Remove the circles with low orange coverage. By removing the circles with low orange coverage, ensure that the remaining circles are closely related to the orange area.

[0032] Step 62: Remove noise circles. By comparing the radius of the current circle with the average radius of all circles, small-sized circles can be effectively filtered out.

[0033] Step 63: Remove the overlapping circles with smaller score values, and retain the overlapping circles that best fit the navel orange according to the score value.

[0034] Preferably, the score is defined as follows:

[0035] score=(mn) / r

[0036] m represents the number of contour points between two blue circles, d f is the contour point distance threshold, where the distance between each blue circle and the red circle is d f =10;

[0037] n represents the number of contour points in the blue circle, and r represents the radius of the red circle.

[0038] Preferably, in step 52, each time a circle is drawn, the points on the new circle are deleted, and the points on the circle are defined as follows: all points between two blue circles are regarded as points on the red circle.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] The method of the present invention combines contour analysis with grid processing strategies, effectively solving the problems of illumination changes, fruit stacking and leaf occlusion faced by the existing technology in navel orange recognition, while improving the recognition speed and accuracy. The performance of experimental data on the standard navel orange image dataset shows that this method not only performs well in recognition accuracy, but also shows good adaptability to navel oranges of different sizes and shapes, opening up a new path for intelligent agricultural applications such as automated picking and grading, indicating that in the future, through continuous optimization of methods, the recognition efficiency and accuracy of navel oranges and other fruits and vegetables can be further improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for describing the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0042] Figure 1 It is a flow chart of the present invention;

[0043] Figure 2 Schematic diagram of removing noise contour of the present invention;

[0044] Figure 3 It is a schematic diagram of the characteristic point range of a circle of the present invention;

[0045] Figure 4 It is a schematic diagram of points on the circle of the present invention;

[0046] Figure 5 It is a schematic diagram of low / high orange coverage circles of the present invention;

[0047] Figure 6 A schematic diagram of a noise circle of the present invention;

[0048] Figure 7 A schematic diagram of overlapping / non-overlapping circles of the present invention;

[0049] Figure 8 This is a schematic diagram of illumination change recognition according to the present invention;

[0050] Fig. 9 This is a schematic diagram of fruit stacking identification in the present invention;

[0051] Fig.10 This is a schematic diagram of leaf occlusion recognition in the present invention. DETAILED DESCRIPTION

[0052] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0053] First, the image is gridded, and then points are randomly selected from the grid to draw circles, and the contour points that the circles pass through are removed to reduce the number of circles, thereby improving the recognition speed of the method for navel oranges. To optimize the recognition results, the NOR method is a navel orange recognition method based on contour analysis and gridding. It removes low-coverage circles and overlapping circles to retain circles that fit the navel orange better, and removes noise circles to avoid interference with the method recognition caused by distant navel oranges.

[0054] See also Figure 1 :A navel orange recognition method based on contour analysis and gridding, comprising at least the following steps:

[0055] S1: For the navel orange image obtained by the picking robot, the orange pixel points are retained using the orange RGB range;

[0056] S2: Draw the contour on the original image based on the retained orange pixels;

[0057] S3: Obtain contour points by removing noise contours;

[0058] In S3, the area of ​​each contour is calculated, and the noise contour is removed according to the contour scale factor;

[0059] The noise profile is defined as follows:

[0060]

[0061] Where S is the noise profile, S countour_maxis the upper limit of the number of contours, It is assumed that the contour scale factor is the noise scale factor;

[0062] 2(a) and Figure 2 (b) shows the comparison before and after removing the noise contour. Figure 2 In (a), S countour_max = 25391. For any contour S = 25, assuming the contour scaling factor Then S is the noise profile, because

[0063] S4: Divide the image into several grids and record the number of initial contour points in each grid;

[0064] S5: Draw circles one by one using contour points and calculate their scores;

[0065] S6: Remove inaccurate circles: circles with low orange coverage, noise circles, and small overlapping circles;

[0066] S7: Draw an accurate circle on the original image to obtain an image containing one or more circles, and identify the position and size of the navel orange through the image.

[0067] Step 5 includes at least the following steps:

[0068] Step 51: Randomly select 3 contour points from different grids to draw a new circle and calculate its score. The score is the accuracy of the circle. The new circle is two blue circles and one red circle, and the red circle is located between the blue circles.

[0069] The definition of score is as follows:

[0070] score=(mn) / r

[0071] m represents the number of contour points between two blue circles, d f is the contour point distance threshold, where the distance between each blue circle and the red circle is d f =10;

[0072] n represents the number of contour points in the blue circle, and r represents the radius of the red circle;

[0073] like Figure 3 As shown in (a), m represents the number of contour points between two blue circles, d f is the contour point distance threshold, where the distance between each blue circle and the red circle is d f =10. Figure 3 As shown in (b), n represents the number of contour points in the blue circle, where the distance between the blue circle and the red circle is 2d f =20. Figure 4 For the red circle in the figure, r=70, m=170, and n=10, then score=(mn) / r=(170-10) / 70≈2.29.

[0074] Step 52: Delete the contour points on the new circle;

[0075] Step 52 is to delete the points on the new circle every time a circle is drawn. The points on the circle are defined as follows: all points between two blue circles are regarded as points on the red circle.

[0076] Figure 4 All points between the two blue circles in are considered as points on the red circle. The center and radius of the red circle are O(500,300) and r=130 respectively. Assume that the circle distance threshold d c =5, for the contour point P1(630,300), Therefore P1 is a point on the circle. For the contour point P2 (640,300), Therefore P2 is not a point on the circle.

[0077] Step 53: Delete the meshes with less than 20% of the initial contour points;

[0078] Step 54: Traverse the grid and repeat steps 51 to 53 until the number of grids is less than 3. When the number of grids is less than 3, it is not enough to support drawing circles. After the loop ends, a set of circles and their scores are obtained.

[0079] Step 6 includes at least the following steps:

[0080] Step 61: Remove the circles with low orange coverage. By removing the circles with low orange coverage, ensure that the remaining circles are closely related to the orange area.

[0081] The definitions of the low / high orange coverage circles are as follows:

[0082] Assume that the coverage threshold θ = 0.8. Figure 5 As shown, for the blue circle c1, a1=20000, b1

[0083] =30000, then a1 / b1=0.67<θ=0.8, so circle c1 is a circle with low orange coverage. For red circle c2, a2=19000, b2=20000, then a2 / b2=0.95>θ=0.8, so circle c2 is a circle with high orange coverage.

[0084] Step 62: Remove noise circles. By comparing the radius of the current circle with the average radius of all circles, small-sized circles can be effectively filtered out.

[0085] The definition of the noise circle is as follows:

[0086] As shown Figure 6 in the figure, r1 = 150 and r2 = 50 are the radii of the red circle c1 and the blue circle c2 respectively. Assuming the radius ratio factor α = 0.9, the average radius r of all circles average =(r1 + r2) / 2 = 100. Since r2 / r average = 50 / 100 = 0.5 < α = 0.9, the blue circle c2 is a noise circle.

[0087] Step 63: Remove the overlapping circles with smaller score values and retain the overlapping circle that best fits the navel orange according to the score value.

[0088] The definitions of overlapping / non - overlapping circles are as follows:

[0089] As shown Figure 7 in the figure, the centers of the left red circle c1, the right red circle c2, and the blue circle c3 are O1(300, 300), O2(500, 300), and O3(460, 340) respectively. Their radii are r1 = 70, r2 = 80, and r3 = 60 respectively. The distance between c1 and c3 is d 13 ≈165 > min(r1, r3)=60, so c1 and c3 are non - overlapping circles. The distance between c2 and c3 is d 23 ≈57 < min(r2, r3)=60, so c2 and c3 are overlapping circles.

[0090] It should be emphasized that:

[0091] 1. Grid - based and random circle drawing

[0092] In the navel orange recognition task, to improve the recognition speed and optimize resource utilization, the present invention introduces a grid - based processing combined with a random circle drawing strategy.

[0093] In the process of randomly selecting contour points, by dividing the image into several uniform grids and then randomly selecting three contour points from three different grids, it helps to control the randomness and avoid the selected three points being concentrated in the same area. At the same time, the grid structure simplifies the subsequent iterative processing and data update, facilitating the tracking and management of feature points.

[0094] In the process of drawing circles, by traversing and comparing the distances between all contour points in the grids passed by the circle and the center of the newly drawn circle, the NOR method can delete the points on the circle from the grid, thus avoiding generating multiple identical circles and quickly reducing the number of contour points, accelerating the process of drawing circles.

[0095] 2. High - precision circle positioning

[0096] To achieve high - precision positioning of navel oranges, the NOR method takes a series of measures in the circle screening stage.

[0097] First, by retaining the circles with high orange coverage and ensuring that the remaining circles are closely related to the orange area, the positions of navel oranges in the image can be accurately reflected, thus effectively improving the positioning accuracy.

[0098] Then, in order to solve the problem of small-sized circle interference that may be generated by navel oranges at a long distance, the NOR method is used to remove the noise circles. By comparing the radius of the small-sized circle with the optimal circle radius threshold, the circles that obviously do not meet the size of navel oranges are filtered out.

[0099] Finally, for the possible circle overlap problem, the NOR method retains the larger circle in Definition 6. For multiple circles drawn on the same navel orange, the circle that best fits the navel orange is retained, so that each circle in the final result set can more accurately reflect the size and position of the navel orange in the image.

[0100] Technical Effects

[0101] 1. Experimental results and analysis

[0102] a. Navel orange identification results

[0103] The present invention uses 417 navel orange images taken from a navel orange planting base in Chongqing as a data set to experiment with the navel orange recognition method. The 417 navel orange images contain 495 navel oranges, and the NOR method draws 484 circles, of which 467 circles are correctly matched with navel oranges. The recognition accuracy is 96.5%, the recall rate is 94.3%, and the F1 value is 95.4%. The experimental results show that the NOR method has high recognition accuracy, can effectively identify navel oranges in images, and can meet the detection requirements of navel orange automatic picking robots in practical applications.

[0104] Table 1 Analysis of test results

[0105]

[0106] b. Navel orange recognition speed

[0107] The automatic navel orange picking robot needs to complete navel orange recognition within a few seconds. Table 2 shows the time distribution of the NOR navel orange recognition method for 417 images. As can be seen from Table 2, the NOR method can complete the recognition of 80% of the images within 4 seconds and can complete the recognition of all images within 8 seconds, which can meet the real-time detection requirements of the automatic navel orange picking robot.

[0108] Table 2 Recognition speed analysis

[0109]

[0110]

[0111] 2. Visualization effect analysis

[0112] Figure 8 - Fig.10 These are the recognition effects of the navel orange recognition method based on contour analysis and gridding in the present invention under conditions of changing illumination, stacked fruits, and leaf occlusion. The experimental results show that the NOR method can accurately locate the position and size of navel oranges in scenes such as facing light, backlighting, single fruit, overlapping fruits, and occlusion by branches and leaves.

[0113] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A navel orange recognition method based on contour analysis and gridding, characterized in that: At least the following steps are included: S1: For the navel orange image obtained by the picking robot, the orange pixel points are retained using the orange RGB range; S2: Draw the contour on the original image based on the retained orange pixels; S3: Obtain contour points by removing noise contours; S4: Divide the image into several grids and record the number of initial contour points in each grid; S5: Draw circles one by one using contour points and calculate their scores; The step 5 at least comprises the following steps: Step 51: randomly select 3 contour points from different grids to draw a new circle, and calculate its score, where the score is the accuracy of the circle. The new circle is two blue circles and one red circle, and the red circle is located between the blue circles. Step 52: Delete the contour points on the new circle; Step 53: Delete the meshes with less than 20% of the initial contour points; Step 54: traverse the grid and repeat steps 51 to 53 until the number of grids is less than 3. When the number of grids is less than 3, it is not enough to support drawing circles. After the loop ends, a set of circles and their scores are obtained. The score is the accuracy of the circle, and the score is defined as follows: score=(mn) / r m represents the number of contour points between two blue circles, d f is the contour point distance threshold, where the distance between each blue circle and the red circle is d f =10; n represents the number of contour points in the blue circle, and r represents the radius of the red circle; The step 52 is to delete the points on the new circle each time a circle is drawn. The points on the circle are defined as follows: All points between two blue circles are considered as points on the red circle. S6: Remove inaccurate circles: circles with low orange coverage, noise circles, and overlapping circles with small score values; S7: Draw an accurate circle on the original image to obtain an image containing one or more circles, and identify the position and size of the navel orange through the image.

2. The navel orange recognition method based on contour analysis and gridding according to claim 1, characterized in that: The step 6 at least comprises the following steps: Step 61: Remove the circles with low orange coverage. By removing the circles with low orange coverage, ensure that the remaining circles are closely related to the orange area. Step 62: Remove noise circles. By comparing the radius of the current circle with the average radius of all circles, small-sized circles can be effectively filtered out. Step 63: Remove the overlapping circles with smaller score values, and retain the overlapping circles that best fit the navel orange according to the score value.

Images