A method for identifying a pear variety and its application

Abstract

The application discloses a pear variety identification method and application thereof, and comprises the following steps: S1, acquiring an original pear fruit image, and constructing an occlusion sample set and a non-occlusion sample set after data processing; S2, performing contour extraction on the pear fruit images in the occlusion sample set and the non-occlusion sample set, and constructing an occlusion identification data set based on the extracted contours; S3, constructing an occlusion identification model based on an SVM, and training the occlusion identification model through the occlusion identification data set; S4, constructing a pear variety identification model based on a convolutional neural network, and training the pear variety identification model through the non-occlusion sample set; S5, inputting a pear fruit image to be identified into the occlusion identification model after contour extraction, identifying whether the pear fruit image is non-occluded, and if yes, performing pear variety identification through the pear variety identification model, and if not, performing pear variety identification based on a sliding window and the pear variety identification model. The application can quickly and accurately identify different varieties of pears.

Description

[0001] This application is a divisional application of application number 202110255780.4, filed on March 9, 2021, with the invention title "A method for identifying pear varieties based on image recognition" at the time of filing. Technical Field

[0002] This invention relates to the field of image recognition technology, and in particular to a method for identifying pear varieties and its application. Background Technology

[0003] As a deciduous tree, the pear is one of the essential fruits in the home. Its flesh is sweet, delicious, crisp, and juicy, and it is also rich in vitamins and other nutrients. There are many varieties of pears, and different varieties have different nutritional components. Common pear varieties in our daily lives include: Ya pear, Snowflake pear, Apple pear, and Italian black pear, each with its own unique characteristics and nutritional value. For example, Ya pear is rich in vitamin B, which can protect the heart, reduce fatigue, enhance myocardial vitality, and lower blood pressure. Snowflake pear contains a large amount of sugar and tannins, which can relieve phlegm and cough and has a therapeutic effect on the throat. Apple pear is different from ordinary fruits; it is sweet and sour, with large, smooth, crisp, and juicy flesh, small core, and an edible portion of up to 86%. It is rich in vitamins such as VB1 and VB2, as well as calcium, phosphorus, and iron, making it highly nutritious. The skin of Italian black pear has the effects of clearing the heart and lungs, reducing internal heat and promoting body fluid production, and nourishing the kidneys and yin. The pear seeds contain lignin, which can dissolve in the intestines to form a gel-like film that combines with cholesterol in the intestines to eliminate cholesterol, and can also treat constipation and prevent osteoporosis in women. The flesh of the pear is sweet and sour, nutritious, and rich in dietary fiber; each pear can meet 40% of the daily dietary fiber requirement.

[0004] Based on the rich nutritional value and excellent taste of pears, their sales are showing a very rapid growth trend. How to quickly and accurately sort different varieties of pears is a technical problem that urgently needs to be solved. Therefore, it is necessary to provide a pear variety identification method based on image recognition to realize the automatic sorting of different varieties of pears. Summary of the Invention

[0005] The purpose of this invention is to provide a method for identifying pear varieties and its application, in order to solve the technical problems existing in the prior art and to enable rapid and accurate identification of different pear varieties.

[0006] To achieve the above objectives, the present invention provides the following solution: The present invention provides a method for identifying pear varieties, comprising:

[0007] S1. Obtain the original pear fruit image and perform data processing on the obtained pear fruit image. Based on the data-processed pear fruit image, construct the occluded sample set and the non-occluded sample set respectively.

[0008] S2. The pear fruit images in the occluded sample set and the non-occluded sample set obtained in step S1 are converted to grayscale. Contours are extracted from the grayscale images obtained by grayscale conversion. An occlusion recognition dataset is constructed based on the extracted contours.

[0009] S3. Construct an occlusion recognition model based on SVM, and train the occlusion recognition model using the occlusion recognition dataset to obtain a trained occlusion recognition model.

[0010] S4. Construct a pear variety recognition model based on a convolutional neural network, and train the pear variety recognition model using the unoccluded sample set constructed in step S1 to obtain a trained pear variety recognition model.

[0011] S5. Obtain the pear fruit image to be identified. Extract the contour of the pear fruit image to be identified in step S2 and input it into the occlusion recognition model trained in step S3. If it is identified as non-occluded, input the pear fruit image to be identified into the pear variety recognition model trained in step S4 to obtain the pear variety recognition result. If it is identified as occluded, obtain the pear variety recognition result based on the sliding window and the trained pear variety recognition model.

[0012] Preferably, the original pear fruit image obtained in step S1 and the pear fruit image to be identified obtained in step S5 are both RGB images.

[0013] Preferably, in step S1, the image processing includes the following steps:

[0014] 1) The acquired pear fruit images are labeled, including the pear variety and whether they are occluded;

[0015] 2) Perform data augmentation on the labeled pear fruit images;

[0016] 3) The acquired pear fruit images are normalized in size.

[0017] Preferably, in step S2, the Canny edge detection algorithm is used to extract the contour of the grayscale image, specifically including:

[0018] Gaussian filtering is used to smooth the grayscale image and remove noise.

[0019] Calculate the intensity gradient of the image to obtain candidate edges;

[0020] Non-maximum suppression is used to refine candidate edges, turning multi-pixel-wide candidate edges into single-pixel-wide edges.

[0021] A dual-threshold method was used to filter and fill the thinned candidate edges to obtain the edge extraction results of the pear in the pear fruit image.

[0022] Preferably, in step S3, the kernel function of the SVM is the Gaussian radial basis function.

[0023] Preferably, in step S4, the convolutional neural network includes an input layer, several convolutional blocks, two fully connected layers, and a softmax layer connected in sequence, with a dropout layer connecting the two fully connected layers.

[0024] Preferably, each convolutional block includes a first convolutional layer, a second convolutional layer, a nonlinear activation function, and a pooling layer connected in sequence, with a batch normalization layer connected between the second convolutional layer and the nonlinear activation function; wherein, the first convolutional layer is used to extract the contour features of the pear fruit image, and the second convolutional layer is used to extract the color features of the pear fruit image.

[0025] Preferably, both the first convolutional layer and the second convolutional layer are followed by a local response value normalization layer for brightness correction.

[0026] Preferably, the pear variety identification model is trained using a backpropagation algorithm, and through training, the weights of the first and second convolutional layers in each convolutional block are updated.

[0027] Preferably, in step S5, the method for obtaining the pear variety identification result based on the sliding window and the trained pear variety identification model includes:

[0028] First, set the size of the sliding window;

[0029] Secondly, in the trained pear variety recognition model, the weight of the first convolutional layer in each convolutional block is set to 0.

[0030] Next, a sliding window is slid across the region enclosed by the outline of the pear fruit image to be identified, and the pear variety identification result at each position is obtained by the trained pear variety identification model. The pear variety with the highest proportion in the identification result is the variety of the pear fruit image to be identified.

[0031] The present invention discloses the following technical effects:

[0032] This invention provides a pear variety identification method based on image recognition. It constructs an occlusion recognition model using SVM and a pear variety identification model using a convolutional neural network. The occlusion recognition model is trained using contour data from a sample set. Since the contour extraction process avoids occluded parts, the contours of occluded pear images are severely distorted. SVM enables fast and accurate occlusion identification. For unoccluded pear images, the trained pear variety identification model is used for identification. For occluded pear images, considering the impact of contour distortion and the uneven color of the pear itself on the identification results, a sliding window and color features from the pear variety identification model are used for pear variety identification. The pear variety with the highest percentage of identification results is selected as the variety of the pear image to be identified. Therefore, fast and accurate variety identification can be achieved for both occluded and unoccluded pear images. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 This is a flowchart of the pear variety identification method based on image recognition according to the present invention. Detailed Implementation

[0035] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0036] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0037] Reference Figure 1 As shown, this embodiment provides a pear variety identification method based on image recognition, including the following steps:

[0038] S1. Obtain the original pear fruit image and perform data processing on the obtained pear fruit image. Based on the data-processed pear fruit image, construct the occluded sample set and the non-occluded sample set respectively.

[0039] The acquired original pear fruit images are RGB images at the fruit ripening stage, including several pear varieties. Each pear variety's fruit images include both occluded and unoccluded images. In this embodiment, the pear varieties include Ya pear, Snowflake pear, Apple pear, and Italian Black pear. Based on factors such as light, angle, distance, and scene during the pear fruit image acquisition process, each variety of pear includes 1000 images. Occlusion includes rotten spots, occlusion by leaves and / or branches, and occlusion by other pear varieties. Image processing includes the following steps:

[0040] 1) The acquired pear fruit images are labeled, including the pear variety and whether they are occluded;

[0041] 2) Perform data augmentation on the labeled pear fruit images. The data augmentation methods include: image rotation, symmetry, scaling, brightness transformation, and adding random noise. Data augmentation can effectively improve the diversity of samples, increase the size of the dataset, and reduce the risk of overfitting.

[0042] 3) Normalize the size of the acquired pear fruit images and adjust all images to a resolution of 224*224.

[0043] Data augmentation and size normalization can effectively ensure the consistency of image input, improve the diversity of sample datasets, and enhance the adaptability of the model.

[0044] (4) Based on the occluded and unoccluded images processed by 1)-3), occluded sample sets and unoccluded sample sets are constructed respectively.

[0045] S2. The pear fruit images in the occluded sample set and the non-occluded sample set obtained in step S1 are converted to grayscale. Contours are extracted from the grayscale images obtained by grayscale conversion. An occlusion recognition dataset is constructed based on the extracted contours.

[0046] In this embodiment, the Canny edge detection algorithm is used to extract contours from the grayscale image. The steps of the Canny edge detection algorithm include:

[0047] Gaussian filtering is used to smooth the grayscale image and remove noise. Noise refers to areas with large grayscale variations, which are easily identified as false edges. Therefore, image smoothing can effectively improve the contour extraction accuracy.

[0048] Calculate the intensity gradient of the image to obtain candidate edges; the gradient is where the gray level changes significantly, and the edge is also where the gray level changes significantly. Therefore, by calculating the intensity gradient of the gray-level image, candidate edges can be obtained.

[0049] Non-maximum suppression is used to refine candidate edges. Usually, the gray-level changes are concentrated. Non-maximum suppression retains the gray-level changes with the largest changes in the gradient direction within the local area, while the others are not retained. This can eliminate a large number of points and turn the multi-pixel-wide edges in the candidate edges into single-pixel-wide edges, that is, turn "fat edges" into "thin edges".

[0050] A dual-threshold method was used to filter and fill the candidate edges after thinning, resulting in the edge extraction results of the pear in the pear fruit image. After non-maximum suppression, there were still many possible edge points. By setting a dual threshold, namely an upper threshold and a lower threshold, pixels with grayscale changes greater than the upper threshold were set as strong edge pixels, those below the lower threshold were discarded, and those between the upper and lower thresholds were set as weak edge pixels. The main edge of the pear in the image was obtained based on the strong edge pixels, and the unclosed positions in the main edge were filled by the weak edge pixels to complete the extraction of the pear edge.

[0051] S3. Construct an occlusion recognition model based on SVM (Support Vector Machine), and train the occlusion recognition model using the occlusion recognition dataset to obtain a trained occlusion recognition model;

[0052] SVM is a generalized linear classifier that performs binary classification of data using supervised learning. It achieves classification based on occlusion and non-occlusion by maximizing the distance between two classes on the hyperplane of the kernel function's parameters. In this embodiment, the SVM uses the Gaussian radial basis function (RBF). The Gaussian RBF is a kernel function with strong locality; its essence is to measure the similarity between samples, allowing similar samples to cluster together better in a similarity space, thus achieving linear separability.

[0053] Since this invention extracts contours based on grayscale variation data, and the depth of the occluded part is much higher than that of the pear body, the occluded part can be avoided during the contour extraction process. As a result, the contour of the occluded pear fruit image is severely distorted, and the occlusion can be quickly and accurately identified by SVM.

[0054] S4. Construct a pear variety recognition model based on a convolutional neural network, and train the pear variety recognition model using the unoccluded sample set constructed in step S1 to obtain a trained pear variety recognition model.

[0055] The convolutional neural network includes an input layer, several convolutional blocks, two fully connected layers, and a softmax layer connected in sequence.

[0056] This embodiment includes three convolutional blocks, wherein each convolutional block includes a first convolutional layer, a second convolutional layer, a nonlinear activation function, and a pooling layer connected in sequence.

[0057] The first convolutional layer is used to extract the contour features of the pear fruit image, which include: fruit contour, stem cavity features, and calyx cavity features; the second convolutional layer is used to extract the color features of the pear fruit image, which include: peel background color, peel overlay color, russeting near the stem cavity, and russeting on the fruit surface.

[0058] The ReLU activation function is used for non-linear activation, and the log activation function is used for pooling layers. Within each convolutional block, the kernel size of the first and second convolutional layers is 7×7 with a stride of 1, while the pooling kernel size is 2×2 with a stride of 10, used for local averaging of the features extracted by the first and second convolutional layers. The number of convolutional kernels increases sequentially across the three convolutional blocks: 64 kernels for the first and second convolutional layers in the first block, 128 kernels for the second block, and 256 kernels for the third block, improving feature representation. Local response value normalization layers are connected after the first and second convolutional layers for brightness correction, effectively avoiding the impact of brightness differences during pear fruit image acquisition on the recognition results.

[0059] In each convolutional block, a batch normalization layer is connected between the second convolutional layer and the non-linear activation function. The batch normalization layer normalizes the data, speeds up the convergence of the model, and improves the stability of the convolutional network. A dropout layer is also connected between the two fully connected layers, which can effectively suppress the overfitting phenomenon of the convolutional neural network and reduce the complexity of the network.

[0060] During the training of a convolutional neural network, a dynamic learning rate adjustment method is used for iterative training. Specifically, the steps include: setting the initial learning rate of the convolutional neural network to 0.01, and iteratively training for 1.5 × 10⁻⁶ cycles. 5 Times, 1.5 × 10 5 During each iteration, at 10... 4 After that, the learning rate will be multiplied by 0.1 to achieve the best effect of feature learning.

[0061] The pear variety identification model is trained using the backpropagation algorithm. Through training, the weights of the first and second convolutional layers in each convolutional block are updated.

[0062] S5. Obtain the pear fruit image to be identified. Extract the contour of the pear fruit image to be identified in step S2 and input it into the occlusion recognition model trained in step S3. If it is identified as non-occluded, input the pear fruit image to be identified into the pear variety recognition model trained in step S4 to obtain the pear variety recognition result. If it is identified as occluded, obtain the pear variety recognition result based on the sliding window and the trained pear variety recognition model.

[0063] The pear fruit image to be identified is an RGB image. Before extracting the contour of the pear fruit image to be identified, it is also normalized in size through step S1 to adjust it to a resolution of 224*224.

[0064] Because occlusion can lead to incomplete pear contour features, resulting in misjudgment or missed judgment, this embodiment uses a sliding window and a trained pear variety recognition model to identify the pear variety from the image of the pear fruit to be identified. The specific method includes:

[0065] First, set the size of the sliding window. In this embodiment, the sliding window size is set to 5×5.

[0066] Secondly, in the trained pear variety recognition model, the weight of the first convolutional layer in each convolutional block is set to 0.

[0067] Next, a sliding window is slid across the region enclosed by the outline of the pear fruit image to be identified, and the pear variety identification result at each position is obtained by the trained pear variety identification model. The pear variety with the highest proportion in the identification result is the variety of the pear fruit image to be identified.

[0068] The present invention has the following technical effects:

[0069] This invention provides a pear variety identification method based on image recognition. It constructs an occlusion recognition model using SVM and a pear variety identification model using a convolutional neural network. The occlusion recognition model is trained using contour data from a sample set. Since the contour extraction process avoids occluded parts, the contours of occluded pear images are severely distorted. SVM enables fast and accurate occlusion identification. For unoccluded pear images, the trained pear variety identification model is used for identification. For occluded pear images, considering the impact of contour distortion and the uneven color of the pear itself on the identification results, a sliding window and color features from the pear variety identification model are used for pear variety identification. The pear variety with the highest percentage of identification results is selected as the variety of the pear image to be identified. Therefore, fast and accurate variety identification can be achieved for both occluded and unoccluded pear images.

[0070] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A method for identifying a variety of pears, characterized by, The identification method is based on image recognition, and comprises the following steps: S1, obtaining an original pear fruit image, and performing data processing on the obtained pear fruit image, and constructing a shielding sample set and a non-shielding sample set based on the data-processed pear fruit image; S2, performing grayscale processing on the pear fruit images in the shielding sample set and the non-shielding sample set obtained in step S1, respectively, performing contour extraction on the grayscale images obtained through the grayscale processing, and constructing a shielding identification data set based on the extracted contours; S3, constructing a shielding identification model based on SVM, and training the shielding identification model through the shielding identification data set to obtain a trained shielding identification model; S4, constructing a pear variety identification model based on a convolutional neural network, and training the pear variety identification model through the non-shielding sample set constructed in step S1 to obtain a trained pear variety identification model; S5, obtaining a pear fruit image to be identified, performing contour extraction on the pear fruit image to be identified through step S2, inputting the pear fruit image to be identified into the trained shielding identification model of step S3, identifying as non-shielding, inputting the pear fruit image to be identified into the trained pear variety identification model of step S4 to obtain a pear variety identification result, and identifying as shielding, and obtaining a pear variety identification result based on a sliding window and the trained pear variety identification model; In step S4, the convolutional neural network comprises an input layer, a plurality of convolutional blocks, two fully connected layers, and a Softmax layer connected in sequence, and a dropout layer is further connected between the two fully connected layers; The obtained original pear fruit image is an RGB image at a fruit ripening stage, comprises a plurality of pear varieties, and the pear fruit image of each pear variety comprises shielding images and non-shielding images; The original pear fruit image obtained in step S1 and the pear fruit image to be identified obtained in step S5 are both RGB images; In step S1, the image processing comprises the following steps: 1) performing marking processing on the obtained pear fruit image, and the marked content comprises a pear variety and whether shielding; 2) performing data enhancement processing on the pear fruit image after the marking processing; 3) performing size normalization processing on the obtained pear fruit image; In step S2, the Canny edge detection algorithm is used to perform contour extraction on the grayscale image, and specifically comprises the following steps: performing image smoothing processing on the grayscale image by using a Gaussian filter to remove noise; calculating the intensity gradient of the image to obtain a candidate edge; performing thinning processing on the candidate edge by using non-maximum suppression to change multiple-pixel-wide edges in the candidate edge into single-pixel-wide edges; performing screening and filling on the thinned candidate edge by using a double-threshold method to obtain an edge extraction result of the pear in the pear fruit image.

2. The method of identifying a pear variety according to claim 1, characterized in that, In step S3, the kernel function of SVM is a Gauss radial basis function.

3. Use of a method for identifying a pear variety in a pear variety, characterized in that, The identification method of any one of claims 1-2 is used for identification.

Images