A small sample set based on deep network pear defect classification method
By constructing a multi-layer feature fusion cascade network of sparse autoencoder (SAE) in pear defect detection, and combining multiple image features, the problems of low accuracy and subjective feature selection in pear defect detection with small sample sets are solved, and efficient pear defect classification and quality evaluation are achieved.
Patent Information
- Application Number
- CN202310483086.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-04
- Publication Date
- 2025-12-16
- Estimated Expiration
- 2043-05-04
AI Technical Summary
Existing technologies for pear defect detection suffer from problems such as large sample requirements, highly subjective feature selection, and lack of theoretical guidance. This leads to low detection accuracy of deep learning models on small sample sets, and pear surface defects are easily confused with fruit stalks and calyxes.
A deep network approach with a small sample set is adopted. Pear images are acquired by a CCD camera, divided into multiple features, and a multi-layer feature fusion cascade network of sparse autoencoder (SAE) is constructed. Features such as RGB, HSV, gray-level co-occurrence matrix, and directional gradient histogram are combined to perform feature fusion and selection, and optimize the feature vector to improve the detection accuracy.
Achieving high-accuracy classification of pear defects on a small sample set solves the problems of insufficient sample size and subjective feature selection, providing a more scientific basis for pear quality evaluation.
Smart Images
Figure CN116486173B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application belongs to the technical field of visual measurement, and in particular, the present application relates to a pear defect classification method based on a deep network under a small sample set. BACKGROUND
[0002] As an important economic fruit, pear is rich in nutrients and has high medicinal value, but the defects on the surface of the pear can easily lead to bacterial growth, making storage difficult, and greatly affecting the taste and quality of the pear. Therefore, detecting the defects on the surface of the pear is of great significance for fruit grading and improving the competitiveness of agricultural products. At present, image-based visual detection technology is increasingly applied to fruit detection.
[0003] The identification and classification of defects in pear images cannot be separated from the design of the classifier, and the precision, stability and speed of the system depend on the selected algorithm. At present, the supervised machine learning classifier is more commonly used in fruit defect detection, such as Support Vector Machine (SVM) and neural network. Deep learning has shown great potential in image analysis. Among them, Tan uses a convolutional neural network (CNN) to identify and diagnose surface lesions of fruits (Multimedia Tools and Applications, 2016, 75(24), 16741-16761). Feng applies a CNN model to detect subtle bruises of winter jujube in pixel-level spectral information, which is shorter in running time and achieves better results compared with SVM (IEEE Access, 2019, 7, 64494-64505).
[0004] Although deep learning has been greatly developed, there are still difficulties. These difficulties are mainly reflected in the following aspects:
[0005] 1. Based on the method of supervised learning, especially deep network, to obtain excellent classification ability, the training process needs a large number of samples. However, in the detection of pear bruising, it will consume a lot of manpower and financial resources to artificially produce a large number of defect samples, and the practicality is not strong.
[0006] 2. In the selection of pear image features, different scholars have great differences, which is highly subjective and lacks relevant theoretical guidance; in the detection of pear defects, the defects on the surface of the pear are easily confused with the pedicel and calyx.
[0007] Therefore, how to construct a deep network suitable for small data sets and improve the accuracy of defect detection has important research significance and practical value for scientific fruit grading. SUMMARY
[0008] The present application provides a kind of small sample set based on deep network pear defect classification method to solve the problems in the above background.
[0009] To achieve the above object, the technical scheme adopted by the present application is as follows: a small sample set based on deep network pear defect classification method, specifically comprising the following steps:
[0010] Step S1: using CCD camera to collect pear images of different defect types on the workbench to establish a defect image database;
[0011] Step S2: using MATLAB software to divide the collected pear images of different defect types into several image blocks with a pixel resolution of 128x128, select one thousand image blocks, and according to the defect classification, the image blocks contain scar area, rot area, brown spot area, flower stem area, healthy area and bruise area; then mark the scar area, rot area, brown spot area, flower stem area, healthy area and bruise area of the image in order of 0, 1, 2, 3, 4, 5 to form a label, and form a mat file as the label of the small sample data set;
[0012] Step S3: extract multiple features of each image block and normalize them;
[0013] Step S4: merge the multiple features of the image blocks in order to form a mat file and form a feature vector;
[0014] Step S5: repeat steps S1 to S4 to extract the feature vector of each image block, form a feature matrix, and merge it with the label data to form a small sample data set; divide the data set into training data set and test data set in the ratio of 8:2;
[0015] Step S6: construct a deep network to train the data set to realize feature fusion and selection;
[0016] Step S7: extract multiple features of the image block and use the trained network to intelligently classify the features.
[0017] Preferably, in step S3, the following steps are included:
[0018] Step S30: extract the average color feature components of each image block in RGB color space, the hue, saturation and brightness components in HSV space, and merge them to form a 100-dimensional RGB, HSV feature vector;
[0019] Step S31: extract texture features based on gray level co-occurrence matrix GLCM, including energy, entropy, contrast and correlation, to form a 4-dimensional texture feature vector;
[0020] P represents a normalized frequency matrix of the gray level co-occurrence matrix, wherein: i, j represent two gray level values of two pixels appearing simultaneously in a certain direction, so P(i, j) represents the probability of the appearance of the two pixels meeting the condition;
[0021] a energy: ;
[0022] b entropy: ;
[0023] c contrast: ;
[0024] d correlation: ;
[0025] wherein: , is the mean, , is the standard deviation;
[0026] Step S32: extract the direction gradient histogram (HOG) feature.
[0027] Preferably, the step S32 specifically comprises the following steps:
[0028] Step S320: first divide the image block into many small connected regions, i.e., cell units, and then collect the gradient or edge direction histogram of each pixel point in the cell unit;
[0029] Step S321: combine each cell unit into a large, spatially connected interval; in this way, the gradient or edge direction histogram of all cells in an interval is concatenated to obtain the direction gradient histogram feature of the interval, forming a 324-dimensional direction gradient histogram feature vector.
[0030] Preferably, the step S6 specifically comprises the following steps:
[0031] A multi-layer feature fusion cascade network based on a sparse auto-encoder (SAE) is constructed, the cascade network contains three auto-encoding networks, the training data set is put into the network for training, the deep structure of the features is mainly learned, feature selection and fusion are performed, and the optimal features are obtained through training, so that the classification accuracy of defects is improved.
[0032] The above technical scheme has the following beneficial effects:
[0033] 1. The pear defect classification method based on a deep network under a small sample set provided by the application fuses various features of an image, and deeply analyzes the image features by using a deep network, so that the feature vector is optimized and has a high accuracy under a small sample set.
[0034] 2, The pear defect classification method based on a deep network under a small sample set provided by the application has deep analysis on small sample image features, studies defect performance from aspects of color, texture and the like, and solves the problems that current defect grading is mostly based on defect area, the index is single, and grading is relatively rough. The application explores the relationship between each feature component and the defect, and establishes a relevant feature vector and a correlation coefficient, thereby providing a more scientific basis for further quality evaluation of the pear. BRIEF DESCRIPTION OF DRAWINGS
[0035] Figure 1 is a technology roadmap;
[0036] Figure 2 is a schematic view in which pear images of different defect types are divided into a plurality of image blocks with a pixel resolution of 128*128;
[0037] Figure 3 is a schematic view of training label making;
[0038] Figure 4 are gray scale images of brown spots, bruises and scars respectively;
[0039] Figure 5 are component images of R, G, B, H, S and V of the brown spots;
[0040] Figure 6 are component images of R, G, B, H, S and V of the scars;
[0041] Figure 7 are component images of R, G, B, H, S and V of the bruises; DETAILED DESCRIPTION
[0042] The specific embodiments of the application will be further described in detail below with reference to the drawings, and the purpose is to help the technical personnel in the art have a more complete, accurate and in-depth understanding of the concept and technical solution of the application, and to help them implement the application.
[0043] As shown in Figures 1 to 7 , the application is a pear defect classification method based on a deep network under a small sample set, which fuses a plurality of features of an image, and deeply analyzes image features by using a deep network to optimize a feature vector, so that the feature vector has a high accuracy under a small sample set.
[0044] The embodiment describes a specific working mode:
[0045] Embodiment 1
[0046] Step S1: A CCD camera is used to collect pear images of different defect types on a workbench, and is used to establish a defect image database;
[0047] Step S2: using MATLAB software to divide the collected pear images of different defect types into a plurality of image blocks with a pixel resolution of 128x128, as shown in FIG. 2, select one thousand image blocks therefrom, and according to the defect classification, the image blocks contain the scar area, rot area, brown spot area, flower stem area, healthy area and bruise area of the image; then mark the scar area, rot area, brown spot area, flower stem area, healthy area and bruise area of the image in order of 0, 1, 2, 3, 4, 5 to form labels, and form a mat file, as shown in FIG. 3, as labels of the small sample data set; Figure 2 Figure 3
[0048] Step S3: extract a plurality of features of each image block and perform normalization;
[0049] Specifically, the following steps are included:
[0050] Step S30: extract the average color feature components of each image block in the RGB color space, the hue, saturation and brightness components in the HSV space, and combine to form a 100-dimensional RGB, HSV feature vector; as shown in FIG. 4, FIG. 5 and FIG. 6, which are the component diagrams of R, G, B and H, S, V of the brown spot, scar and bruise respectively; Figure 5 、 Figure 6 、 Figure 7
[0051] Step S31: extract texture features based on the gray level co-occurrence matrix GLCM, including energy, entropy, contrast and correlation, to form a 4-dimensional texture feature vector;
[0052] Let P represent the normalized frequency matrix of the gray level co-occurrence matrix, where i, j represent two gray level values that simultaneously appear in two pixels in a certain direction, so P(i, j) represents the probability of the occurrence of the two pixels meeting this condition;
[0053] a energy: ;
[0054] b entropy: ;
[0055] c contrast: ;
[0056] d correlation: ;
[0057] where: , is the mean, , is the standard deviation;
[0058] Step S32: extract the HOG feature; specifically, the following steps are included:
[0059] Step S320: first divide the image block into many small connected regions, i.e. cell units, and then collect the gradient or edge direction histogram of each pixel point in the cell unit;
[0060] Step S321: combine each cell unit into a large, spatially connected interval; thus, the direction gradient histogram feature of the interval is obtained by concatenating the direction gradient histograms of all cells in the interval, forming a 324-dimensional direction gradient histogram feature vector;
[0061] Step S4: merge the various features of the image block into a mat file in order, forming a feature vector, and a total of 629-dimensional feature vector is formed;
[0062] Step S5: repeat steps S1 to S4 to extract the feature vector of each image block, form a 628x1000-dimensional feature matrix, and merge it with the label data to form a small sample data set; divide the data set into a training data set and a test data set in a ratio of 8:2;
[0063] Step S6: construct a deep network to train the data set to realize feature fusion and selection; specifically including the following steps:
[0064] A multi-layer feature fusion cascade network based on sparse autoencoder SAE is constructed, and the cascade network includes three autoencoding networks, such as the deep network in Figure 1 The training data set is put into the network for training, mainly learning the deep structure of the features, and performing feature selection and fusion, and the optimal features are obtained through training to improve the classification accuracy of defects;
[0065] Step S7: extract various features of the image block, and use the trained network to intelligently classify the features.
[0066] The above has been described by way of example in conjunction with the drawings, and it is obvious that the specific implementation of the present application is not limited by the above manner, as long as various non-essential improvements are made using the method concept and technical solution of the present application; or without improvement, the above concept and technical solution of the present application is directly applied to other occasions, which is within the protection scope of the present application.
Claims
1. A deep network-based pear defect classification method under a small sample set, characterized in that: Specifically comprising the following steps: Step S1: using a CCD camera to collect pear images of different defect types on a workbench to establish a defect image database; Step S2: using MATLAB software to divide the pear images of different defect types into a plurality of image blocks with a pixel resolution of 128x128, selecting one thousand image blocks therefrom, and according to defect classification, the image blocks contain scar areas, rot areas, brown spot areas, flower stem areas, healthy areas and bruise areas; then marking the scar areas, rot areas, brown spot areas, flower stem areas, healthy areas and bruise areas of the images in order of 0, 1, 2, 3, 4 and 5 to form labels, and forming a mat file as labels of a small sample data set; Step S3: extracting a plurality of features of each image block and performing normalization; Step S4: merging the plurality of features of the image blocks in order to form a mat file as a feature vector; Step S5: repeating steps S1 to S4 to extract the feature vector of each image block, forming a feature matrix, and merging the feature matrix with the label data to form a small sample data set; dividing the data set into a training data set and a test data set in a ratio of 8:2; Step S6: constructing a deep network to train the data set to realize feature fusion and selection; Step S7: extracting a plurality of features of the image block, and using the trained network to intelligently classify the features; In step S3, the following steps are specifically included: Step S30: extracting each image block of the average color feature component of each channel of the RGB color space, the hue, saturation and brightness components of the HSV space, and merging to form a 100-dimensional RGB, HSV feature vector; Step S31: extracting texture features based on a gray level co-occurrence matrix GLCM, including energy, entropy, contrast and correlation, to form a 4-dimensional texture feature vector; Let P represent the normalized frequency matrix of the gray level co-occurrence matrix, where i, j represent two gray level values that simultaneously appear in two pixels in a certain direction, so P(i, j) represents the probability of the occurrence of two pixels meeting this condition; a energy: ; b entropy: ; c contrast: ; d correlation: ; wherein: , is the mean, , is the standard deviation; Step S32: extracting a histogram of oriented gradients HOG feature; In step S32, the following steps are specifically included: Step S320: first dividing the image block into a plurality of small connected regions, i.e. cell units, and then collecting the gradient or edge direction histogram of each pixel point in the cell unit; Step S321: combining each cell unit into a large, spatially connected interval; in this way, the gradient or edge direction histogram of all cells in an interval is concatenated to obtain the direction gradient histogram feature of the interval, forming a 324-dimensional direction gradient histogram feature vector.
2. The method according to claim 1, wherein the method is characterized by: In step S6, the following steps are specifically included: A multi-layer feature fusion cascade network based on a sparse autoencoder SAE is constructed, the cascade network contains three autoencoding networks, the training data set is put into the network for training, the deep structure of the features is mainly learned, feature selection and fusion are performed, and the optimal features are obtained through training to improve the classification accuracy of defects.