A fruit and vegetable grading and recognition deep learning model training sample image labeling method

Abstract

The application discloses a kind of fruit and vegetable hierarchical recognition deep learning model training sample image annotation methods, comprising the following steps: obtaining fruit and vegetable image;The fruit and vegetable image obtained is preprocessed;The image after pre-processing is cut into small pieces, and a plurality of small pieces of cut image is obtained;Overall labeling is carried out to the cut image, wherein: the cut image without fruit and vegetable defect and without black spot is labeled as the first type;The cut image with fruit and vegetable defect is labeled as the second type;The cut image without fruit and vegetable defect but with black spot is labeled as the third type.The present application is labeled with each cut image as a unit, in the labeling process, fruit and vegetable defects, black spots or fruit stem do not need to be found one by one and labeled on the image, only qualitative determination and labeling whether cut image exists fruit and vegetable defect or black spot, this kind of labeling mode significantly simplifies the labeling step, reduces the labeling workload, reduces the labeling work intensity, while greatly improves the labeling efficiency.

Description

Technical Field

[0001] This invention relates to the field of automatic fruit and vegetable grading technology, and in particular to an image annotation method for training samples of a deep learning model for fruit and vegetable grading and recognition. Background Technology

[0002] In recent years, many scholars at home and abroad have studied machine vision-based methods for inspecting the quality of agricultural products. However, traditional machine learning methods for grading the surface quality of fruits and vegetables have drawbacks, such as the need for detailed human knowledge, interference from human judgment factors, long computation time, and low accuracy when processing large amounts of data.

[0003] With the development of science and technology, deep learning has gradually emerged and been applied to image recognition. Complex network structures and data training models not only reduce workload but also improve detection speed and accuracy. Labeling, as one of the tasks of deep learning, directly determines the accuracy of the trained model. In fruit and vegetable grading, the labeling work for existing model training is basically done manually, which is very time-consuming and labor-intensive. Especially for some fruits and vegetables such as kumquats and pears, in addition to defects such as scabs and bruises, their surfaces also have stems and numerous black spots. Both stems and black spots are easily misidentified as defects during labeling. Furthermore, to achieve high recognition accuracy, a large number of black spot samples are needed during model training. Therefore, the labeling process requires point-by-point differentiation and labeling, which greatly increases the workload and difficulty of labeling. This makes the labeling work very intensive, eye-straining, and labor-intensive, resulting in low labeling efficiency and low accuracy, thus affecting the model training results. Summary of the Invention

[0004] The purpose of this invention is to provide an image annotation method for training samples of a deep learning model for fruit and vegetable grading and recognition.

[0005] The technical solution to achieve the purpose of this invention is: an image annotation method for training samples of a deep learning model for fruit and vegetable grading and recognition, comprising the following steps:

[0006] S1. Acquire images of fruits and vegetables;

[0007] S2. Preprocess the fruit and vegetable images obtained in step S1. The preprocessing operation includes image segmentation: extracting the fruit and vegetable parts of interest in the image and removing the background parts that are not useful for image recognition.

[0008] S3. Cut the preprocessed image from step S2 into small pieces to obtain several small piece images;

[0009] S4. Perform overall annotation on the segmented images from step S3, where:

[0010] Cut images of fruits and vegetables without defects and without black spots are labeled as Category I;

[0011] Images of cut pieces of fruits and vegetables with defects are labeled as Category II;

[0012] Images of cut fruits and vegetables without defects but with black spots are labeled as Category III;

[0013] An image with one or more grayscale patches is considered an image with black spots; conversely, an image without any grayscale patches is considered an image without black spots. An image with any of the following fruit and vegetable defects—scabs, wrinkles, punctures, and bruises—is considered an image with fruit and vegetable defects; conversely, an image without any of these defects is considered an image without fruit and vegetable defects.

[0014] Furthermore, in step S3, the step of cutting the image preprocessed in step S2 into small pieces is to perform fan-shaped segmentation on the image preprocessed in step S2, using the center point of the image as the center of the circle.

[0015] Furthermore, in step S3, the step of cutting the image preprocessed in step S2 into small pieces is to cut the image preprocessed in step S2 into pieces in a 256*256 matrix form.

[0016] Furthermore, the specific steps in step S3 to cut the image preprocessed in step S2 into small pieces are to use Python language and call the OpenCV machine vision library to program and implement the automatic image slicing function.

[0017] Furthermore, the image segmentation processing method in step S2 is any one of threshold-based segmentation, region-based segmentation, edge-based segmentation, and superpixel segmentation.

[0018] Furthermore, the image segmentation processing method in step S2 is a threshold-based segmentation method, which adopts the H component threshold segmentation method based on the HSV model.

[0019] Furthermore, the preprocessing operations in step S2 also include binarization and noise reduction, with image segmentation, binarization, and noise reduction performed sequentially.

[0020] Furthermore, noise reduction is achieved by determining the area size during the noise reduction process. Noise reduction can be achieved using either expansion / contraction methods or by determining the area size.

[0021] Further, step S1 involves acquiring fruit and vegetable images from multiple angles using multiple cameras. After acquiring the fruit and vegetable images, the number of images is increased using data augmentation methods, including cropping, flipping, and random cropping.

[0022] Furthermore, the fruit and vegetable defects in step S4 are any one of scabs, wrinkles, punctures, or bruises.

[0023] This invention discloses an image annotation method for training samples of a deep learning model for fruit and vegetable grading and recognition. The method divides fruit and vegetable images into multiple small blocks, forming segmented images, each with a self-defined boundary region. During annotation, all content within the segmented image region is used as the annotation target, and the segmented image is holistically labeled. Unlike traditional methods, this annotation method does not use specific features such as fruit and vegetable defects, black spots, or stems as annotation targets, but rather annotates each segmented image individually. During annotation, it is unnecessary to search for and annotate each defect, black spot, or stem on the image; only a qualitative judgment and annotation of the presence of defects or black spots in the segmented image is required. This annotation method avoids the tedious process of quantitatively annotating each defect, black spot, or stem in traditional methods, significantly simplifying the annotation steps, reducing the workload, lowering the intensity of annotation work, and greatly improving annotation efficiency.

[0024] Fruit stems are typically few in number and their image features are generally uniform, thus causing minimal interference with defect identification. In contrast, black spots on fruits and vegetables are numerous and varied in image features and shape, significantly interfering with defect identification. Based on these characteristics, the image annotation method for training samples of the deep learning model for fruit and vegetable grading and identification proposed in this invention selects "fruit and vegetable defects" and "black spots" as two core judgment indicators when annotating segmented images in situations where fruit and vegetable images simultaneously possess three features: defects, black spots, or stems. The annotation results are then categorized into three types. This labeling method, on the one hand, does not use "fruit stems" as a criterion, because the presence or absence of fruit stems does not affect the quality of fruits and vegetables, and therefore their absence will not affect the labeling and classification results. This design not only meets the actual needs of fruit and vegetable quality evaluation, but also avoids the added complexity caused by labeling "fruit stems." Compared with classification methods that use three criteria—"fruit and vegetable defects," "black spots," and "fruit stems"—this invention uses only two criteria: "fruit and vegetable defects" and "black spots," significantly reducing the complexity of labeling, simplifying the manual labeling process, and improving labeling efficiency. On the other hand, in the labeling and classification, this invention classifies sliced ​​images without fruit and vegetable defects but with black spots into a separate category, specifically for deep learning models to train for black spot recognition. This design allows the model to learn and train separately for black spots, effectively avoiding the model misclassifying black spots as fruit and vegetable defects, significantly improving the model's accuracy in recognizing black spots, and further improving the overall accuracy and reliability of fruit and vegetable grading and recognition. Through this optimization, this invention ensures labeling efficiency while also considering the accuracy and practicality of model training.

[0025] This invention breaks through the traditional point-by-point annotation model and achieves "one-time annotation, full element coverage", which greatly improves the efficiency and economy of data annotation while ensuring the accuracy of subsequent model detection. Attached Figure Description

[0026] Figure 1 This is a flowchart of the image annotation method for training samples of the deep learning model for fruit and vegetable grading and recognition according to the present invention;

[0027] Figure 2 This is the original image of the fruit and vegetable in the image annotation method for training samples of the deep learning model for fruit and vegetable grading and recognition in this invention;

[0028] Figure 3 yes Figure 2 The image obtained after image segmentation processing;

[0029] Figure 4 yes Figure 3 The image obtained after binarization;

[0030] Figure 5 yes Figure 4 The image obtained after noise reduction processing;

[0031] Figure 6 yes Figure 2 The final image obtained after preprocessing;

[0032] Figure 7 It is the image obtained by cutting the preprocessed image into blocks in a matrix format;

[0033] Figure 8 This is a comparison chart of the recognition accuracy of three models, ResNet50, VGG16, and AlexNet, under different numbers of blocks in the matrix format.

[0034] Figure 9 This is a schematic diagram of sector-like segmentation performed on a preprocessed image;

[0035] Figure 10 It is a single-block image obtained by performing fan-shaped segmentation on the preprocessed image. Detailed Implementation

[0036] The preferred embodiment of the image annotation method for training samples of the deep learning model for fruit and vegetable grading recognition of the present invention will be described in detail below with reference to the accompanying drawings:

[0037] like Figure 1 As shown, an image annotation method for training samples of a deep learning model for fruit and vegetable grading and recognition, taking kumquat as an example, includes the following steps:

[0038] S1. Acquire images of fruits and vegetables;

[0039] S2. Preprocess the fruit and vegetable images obtained in step S1. The preprocessing operation includes image segmentation: extracting the fruit and vegetable parts of interest in the image and removing the background parts that are not useful for image recognition.

[0040] S3. Cut the preprocessed image from step S2 into small pieces to obtain several small piece images;

[0041] S4. Perform overall annotation on the segmented images from step S3, where:

[0042] Images of cut fruits and vegetables without defects or black spots are labeled as Category 1 and marked as "good";

[0043] Images of cut fruits and vegetables with defects are labeled as Category 2 and marked as "bad";

[0044] Images of cut pieces of fruits and vegetables without defects but with black spots are labeled as Category 3 and denoted as "black spots";

[0045] An image with one or more grayscale patches is considered an image with black spots; conversely, an image without any grayscale patches is considered an image without black spots. An image with any of the following fruit and vegetable defects—scabs, wrinkles, punctures, and bruises—is considered an image with fruit and vegetable defects; conversely, an image without any of these defects is considered an image without fruit and vegetable defects.

[0046] This invention provides an image annotation method for training samples in a deep learning model for fruit and vegetable grading recognition. The method annotates fruit and vegetable images so that the annotated images can be used as training samples to train a deep learning model for fruit and vegetable grading recognition. In the fruit and vegetable image annotation process, images of the fruits and vegetables are first captured using a camera. The acquired fruit and vegetable images are shown below. Figure 2 As shown.

[0047] This invention discloses an image annotation method for training samples of a deep learning model for fruit and vegetable grading recognition. To obtain complete features of fruits and vegetables and improve the accuracy of grading recognition, images of fruits and vegetables can be acquired using multiple cameras from multiple angles, such as using three or more cameras to capture images from the top and sides. After acquiring the images, data augmentation processing can be performed, including but not limited to image cropping, flipping, and random cropping, to increase the diversity of training samples and improve the accuracy of model training. The data augmentation methods include cropping, flipping, and random cropping.

[0048] This invention provides an image annotation method for training samples of a deep learning model for fruit and vegetable grading recognition, such as... Figure 2 In the obtained fruit and vegetable images shown, the parts to be labeled are the fruit and vegetable portions, i.e. Figure 2The yellow kumquat portion is highlighted, while the background outside the fruit and vegetable area is irrelevant and should not be included in the labeling process. After acquiring the fruit and vegetable images, preprocessing is performed, including image segmentation. The segmented image is shown below. Figure 3 As shown, only the fruit and vegetable portion is retained, while the background portion is replaced with black.

[0049] This invention provides an image annotation method for training samples of a deep learning model for fruit and vegetable grading recognition. By segmenting the image, the method extracts the fruit and vegetable parts of interest and removes background parts that are not useful for image recognition. This allows the segmented image to focus on the fruit and vegetable parts, avoids background interference, ensures high-quality training samples, provides a reliable data foundation for model training, and enables the model to focus more on the extraction of fruit and vegetable features, thereby improving the accuracy of the training model for grading recognition.

[0050] The image annotation method for training samples of the deep learning model for fruit and vegetable grading identification of the present invention includes an image segmentation method in step S2, which may be, but is not limited to, a threshold-based segmentation method, a region-based segmentation method, an edge-based segmentation method, or a superpixel segmentation method.

[0051] The image annotation method for training samples of the deep learning model for fruit and vegetable grading recognition of this invention preferably uses a threshold-based segmentation method in step S2. Compared with region-based, edge-based, and superpixel-based segmentation methods, the threshold-based segmentation method only needs to calculate the grayscale value of pixels, without the need for complex region or edge analysis. The algorithm is simple, fast, and computationally efficient, significantly reducing image recognition time and saving computational resources. Furthermore, this method has lower computational resource requirements and can run on resource-constrained devices (such as embedded systems). In addition, it mainly relies on threshold selection, involves fewer parameters, and is easy to adjust and optimize.

[0052] This invention discloses an image annotation method for training samples of a deep learning model for fruit and vegetable grading and recognition. Specifically, the threshold-based segmentation method involves converting the image to grayscale and then performing threshold segmentation using a grayscale histogram. This method divides pixels in the image into foreground (fruit and vegetable portion) and background by setting one or more grayscale thresholds. The specific implementation process is as follows: Based on the color characteristics of the fruit and vegetable image (e.g., the yellow of kumquats), one or more grayscale thresholds are selected. The thresholds can be manually set, determined automatically based on histogram analysis, or using an adaptive algorithm (e.g., the Otsu algorithm); the grayscale value of each pixel in the image is compared with the threshold. If the pixel's grayscale value is greater than the threshold, it is classified as foreground (fruit and vegetable portion); otherwise, it is classified as background; the pixel values ​​of the background portion are set to fixed values ​​(e.g., 0 or 255).

[0053] The image annotation method for training samples of the deep learning model for fruit and vegetable grading recognition in this invention, based on the threshold segmentation method, can also be based on the HSV model to achieve threshold segmentation of a certain component. Specifically, the original RGB image is converted into an HSV image. HSV images can better separate color information and reduce the impact of lighting changes on the segmentation results. HSV images include three components: hue, saturation, and value. Taking threshold segmentation of the hue component (H component) as an example, the H component (hue) is extracted from the HSV image. The value range of the H component is usually from 0° to 360°, representing the type of color (such as red, green, blue, etc.). According to the color characteristics of fruits and vegetables, the threshold range of the H component is set. For example, for yellow kumquats, the threshold range of the H component can be set to 20° to 40°. Each pixel in the image is traversed to determine whether its H component is within the set threshold range. If the H component is within the threshold range, the pixel is classified as foreground (fruit and vegetable part). If the H component is not within the threshold range, the pixel is classified as background. The pixel values ​​of the background part are set to fixed values ​​(such as 0 or 255).

[0054] This invention presents an image annotation method for training samples of a deep learning model for fruit and vegetable grading and recognition. However, the process of converting the image to grayscale and then using a grayscale histogram for threshold segmentation loses color information, relying solely on brightness. This makes the method sensitive to changes in lighting conditions, as brightness variations directly affect grayscale values, leading to unstable segmentation results and difficulty in handling scenes with complex colors or overlapping brightness distributions. In contrast, threshold segmentation using the HSV model preserves color information. Segmentation is performed using three components: hue, saturation, and value. The HSV model separates value from color information (hue and saturation), allowing for adjustments to hue and saturation to mitigate the impact of lighting changes. Furthermore, it offers greater flexibility by selecting different components for segmentation, enabling accurate target segmentation based on color features.

[0055] This invention provides an image annotation method for training samples of a deep learning model for fruit and vegetable grading recognition. When using the HSV model for image segmentation, the H component (hue, saturation, and value) effectively distinguishes fruits and vegetables of different colors, making it particularly suitable for scenes with distinct color features (such as yellow kumquats and red apples). Therefore, this invention's image annotation method for training samples of a deep learning model for fruit and vegetable grading recognition, combined with the distinctive color features of fruit and vegetable images such as kumquat images, preferably employs a threshold-based segmentation method using the HSV model's H component thresholding.

[0056] The image annotation method for training samples of the deep learning model for fruit and vegetable grading recognition of this invention preferably includes, in step S2, a preprocessing operation that further includes binarization and noise reduction. These three operations are performed sequentially in the order of image segmentation → binarization → noise reduction to ensure the accuracy and robustness of the segmentation results. Specifically, the segmented image is binarized to convert it into a black and white binary image. The binarized image is shown below. Figure 4 As shown, the purpose of binarization is to further clarify the boundary between the foreground and background, facilitating subsequent noise reduction processing. Then, noise reduction processing is performed on the binarized image to remove small portions of noise. The noise-reduced image is shown below. Figure 5 As shown, the purpose of noise reduction is to improve image quality and reduce the interference of noise on subsequent annotation and model training. After processing in the order of image segmentation → binarization → noise reduction, the final preprocessed image is shown below. Figure 6 As shown.

[0057] This invention provides an image annotation method for training samples of a deep learning model for fruit and vegetable grading recognition. It clarifies the boundary between the fruit / vegetable portion and the background through binarization, avoiding the influence of blurred areas on the segmentation result. Noise reduction removes minor noise, ensuring a clean and complete segmentation result and enhancing image quality. The high-quality image segmentation results after binarization and noise reduction provide a reliable data foundation for subsequent annotation and model training.

[0058] This invention discloses an image annotation method for training samples of a deep learning model for fruit and vegetable grading recognition. During noise reduction, it can employ either dilation / contraction noise reduction or noise reduction based on area size. The dilation / contraction method uses morphological operations (such as opening and closing operations) to dilate and shrink the image to remove small noise points or fill small holes. The area-based noise reduction method uses connected component analysis to statistically analyze the area of ​​each connected region in the image and filters out target and noise regions based on a set area threshold. Since dilation / contraction noise reduction alters the fruit area, noise reduction based on area size can remove noise without changing the shape and area of ​​the target region. This is particularly suitable for scenarios requiring high accuracy in fruit area measurement, and by combining connected component analysis and area threshold filtering, the noise reduction target can be achieved quickly. Therefore, preferably, the area-based noise reduction method is used during noise reduction.

[0059] This invention discloses an image annotation method for training samples of a deep learning model for fruit and vegetable grading and recognition. After preprocessing the fruit and vegetable images, the preprocessed images from step S2 are cut into small blocks to obtain several small block images. Then, each block image is labeled holistically, wherein: block images without fruit and vegetable defects and without black spots are labeled "good"; block images with fruit and vegetable defects are labeled "bad"; and block images without fruit and vegetable defects but with black spots are labeled "black spots".

[0060] The image annotation method for training samples of a deep learning model for fruit and vegetable grading recognition according to the present invention, in step S3, the process of cutting the image preprocessed in step S2 into small pieces, is firstly implemented by cutting the image preprocessed in step S2 into pieces in a matrix form, such as... Figure 7 As shown, when slicing an image, the size of the slices directly affects the recognition performance. The finer the slices, the more detailed the information in the sliced ​​image can be described, and the larger the receptive field of features such as fruit and vegetable defects or black spots in each sliced ​​image is. However, the dimension of feature extraction will increase exponentially, thereby increasing the time complexity of the algorithm. At the same time, overly fine slices will also make the sliced ​​images too sparse and lose statistical features, leading to overfitting and reducing the recognition rate.

[0061] This invention discloses an image annotation method for training samples of a deep learning model for fruit and vegetable grading recognition. The method involves segmenting the images into blocks using matrices of 64*64, 128*128, and 256*256 respectively. The annotated images are then used to train three models: ResNet50, VGG16, and AlexNet. The recognition accuracy of the three models after training is shown below. Figure 8 As shown. By Figure 8 It is easy to see that the trained model achieves higher recognition accuracy after being divided into blocks in a 256*256 matrix format. Therefore, in the image annotation method for training samples of the deep learning model for fruit and vegetable grading recognition of this invention, preferably, in step S3, the first implementation of the process of dividing the preprocessed image from step S2 into small blocks involves dividing the preprocessed image from step S2 into blocks in a 256*256 matrix format. With 256*256 block images, both the amount of annotation and the accuracy of annotation are balanced. While ensuring that the amount of annotation is not excessive, the receptive field of targets such as fruit and vegetable defects or black spots within the block image is still relatively large, resulting in a relatively high annotation accuracy.

[0062] The image annotation method for training samples of the deep learning model for fruit and vegetable grading recognition of this invention, in step S3, the process of cutting the image preprocessed in step S2 into small pieces, has a second implementation method: using the center point of the image as the center, the image preprocessed in step S2 is divided into fan-shaped segments, such as... Figure 9As shown. Each segmented image block includes two cutting edges. The intersection of these two cutting edges is the center point of the image after S2 preprocessing. The center point of the image after S2 preprocessing falls on the fruit and vegetable portion. The segmented single-block image is shown below. Figure 10 As shown.

[0063] The image annotation method for training samples of the deep learning model for fruit and vegetable grading recognition of the present invention, in step S3, the process of cutting the image after preprocessing in step S2 into small blocks can be performed by cutting blocks in a matrix form or by performing sector-like segmentation.

[0064] When segmenting in a matrix format, each segmented image lacks specific features. Some segments may consist entirely of fruits and vegetables or entirely of background, while others may contain both. The content of the segments is disorganized, making it difficult for staff to form a unified and clear understanding of each segment during annotation. In particular, the black dots on the background and fruits and vegetables are all black, making them easy to confuse and mislabel. Furthermore, if a black dot on a fruit or vegetable segment happens to appear on the cutting line, it is usually identified as background because it is located at the edge of the segmented image, resulting in the black dots on the cutting line not being labeled, leading to omissions and low annotation accuracy.

[0065] When using a fan-shaped segmentation method, each segmented image has both a fruit / vegetable portion and a background portion. The fruit / vegetable portion extends outward from the central corner of the two cutting edges of the segmented image and is located in the central corner region of the segmented image. The background portion is located outside the segmented image. Each segmented image has the same and uniform characteristics. These uniform characteristics allow staff to form a clearer understanding and make clear definitions of the content to be labeled, greatly facilitating labeling and improving labeling efficiency. In addition, with the characteristic that "the fruit / vegetable portion is located in the central corner region of the segmented image and the background portion is located outside the segmented image," the region between the fruit / vegetable portion and the background portion is clearly defined. Whether it is a defect in the fruit / vegetable with a different color or a black dot that is the same black as the background portion, it can be clearly distinguished in the segmented image and accurately labeled. Even if black images appear on the cutting edge of the cut image, the clear division between the fruit / vegetable part and the background part can give the black image on the cutting line a clear area classification. If the black image is on the fruit / vegetable part, it is a black dot on the surface of the fruit / vegetable. If the black image is on the background part, it is the background. This prevents the black dots on the fruit / vegetable part that appear on the cutting line from being mistaken for the background, which greatly improves the accuracy of the annotation.

[0066] This invention discloses an image annotation method for training samples of a deep learning model for fruit and vegetable grading recognition. Step S3, which involves slicing the preprocessed image from step S2 into small blocks, can be performed using Python and the OpenCV machine vision library for automatic image slicing. By setting and adjusting slicing parameters such as the number of rows, columns, or the cutting angle under a fan-shaped slicing pattern, the size and number of blocks can be flexibly controlled. It can perform both uniform and non-uniform slicing.

[0067] This invention discloses an image annotation method for training samples of a deep learning model for fruit and vegetable grading and recognition. The method divides fruit and vegetable images into multiple small blocks, forming segmented images, each with a self-defined boundary region. During annotation, all content within the segmented image region is used as the annotation target, and the segmented image is holistically labeled. Unlike traditional methods, this annotation method does not use specific features such as fruit and vegetable defects, black spots, or stems as annotation targets, but rather annotates each segmented image individually. During annotation, it is unnecessary to search for and annotate each defect, black spot, or stem on the image; only a qualitative judgment and annotation of the presence of defects or black spots in the segmented image is required. This annotation method avoids the tedious process of quantitatively annotating each defect, black spot, or stem in traditional methods, significantly simplifying the annotation steps, reducing the workload, lowering the intensity of annotation work, and greatly improving annotation efficiency.

[0068] Fruit stems are typically few in number and their image features are generally uniform, thus causing minimal interference with defect identification. In contrast, black spots on fruits and vegetables are numerous and varied in image features and shape, significantly interfering with defect identification. Based on these characteristics, the image annotation method for training samples of the deep learning model for fruit and vegetable grading and identification proposed in this invention selects "fruit and vegetable defects" and "black spots" as two core judgment indicators when annotating segmented images in situations where fruit and vegetable images simultaneously possess three features: defects, black spots, or stems. The annotation results are then categorized into three types. This labeling method, on the one hand, does not use "fruit stems" as a criterion, because the presence or absence of fruit stems does not affect the quality of fruits and vegetables, and therefore their absence will not affect the labeling and classification results. This design not only meets the actual needs of fruit and vegetable quality evaluation, but also avoids the added complexity caused by labeling "fruit stems." Compared with classification methods that use three criteria—"fruit and vegetable defects," "black spots," and "fruit stems"—this invention uses only two criteria: "fruit and vegetable defects" and "black spots," significantly reducing the complexity of labeling, simplifying the manual labeling process, and improving labeling efficiency. On the other hand, in the labeling and classification, this invention classifies sliced ​​images without fruit and vegetable defects but with black spots into a separate category, specifically for deep learning models to train for black spot recognition. This design allows the model to learn and train separately for black spots, effectively avoiding the model misclassifying black spots as fruit and vegetable defects, significantly improving the model's accuracy in recognizing black spots, and further improving the overall accuracy and reliability of fruit and vegetable grading and recognition. Through this optimization, this invention ensures labeling efficiency while also considering the accuracy and practicality of model training.

[0069] Taking kumquat images as an example, traditional annotation software is used to annotate kumquat images point by point. Since the number of black dots varies among different kumquat images, the time required is approximately 1-2.5 minutes. Annotating 50 kumquat images would take about 60 minutes, with an average annotation time of 1.2 minutes per image. However, when using the image annotation method of the fruit and vegetable grading recognition deep learning model training samples of this invention to annotate 50 kumquat images, the images are divided into 256*256 blocks, and after removing the background, 1056 sliced ​​images are obtained. Annotating these 1056 sliced ​​images takes 27 minutes. The average time spent annotating each of the 50 kumquat images is 0.54 minutes. Compared to the traditional average annotation time of 1.2 minutes per block, the annotation time of this invention is significantly reduced, and the annotation efficiency is greatly improved.

[0070] This invention requires only image block-level annotation, breaking through the traditional, time-consuming pixel-by-pixel annotation (drawing each black dot or defect bit by bit) model for each training image. It achieves "one-time annotation, full element coverage", which greatly improves the efficiency and economy of data annotation while ensuring the accuracy of subsequent model detection.

[0071] This invention discloses an image annotation method for training samples of a deep learning model for fruit and vegetable grading and recognition. Before image segmentation, the fruit and vegetable images are preprocessed to extract the parts of interest and remove background parts that are useless for image recognition. Removing the background reduces the amount of image data, lowering the computational complexity of subsequent segmentation and annotation, thereby improving overall processing efficiency. Furthermore, by extracting the fruit and vegetable parts, background noise is avoided from interfering with model training, allowing the model to focus more on the extraction and learning of fruit and vegetable features, improving recognition accuracy. Moreover, removing the background makes the annotation targets clearer, reducing unnecessary annotation areas and lowering the workload and time cost of manual annotation. The preprocessed image data is cleaner and more focused, helping the model to learn key features of fruits and vegetables (such as defects and black spots) more efficiently, thereby improving the model's generalization ability and recognition accuracy. Furthermore, removing useless background reduces the image file size, saving storage space and reducing computational resource consumption during training. Finally, by removing background interference, the model can better adapt to fruit and vegetable images in different environments, enhancing its adaptability to complex scenes.

[0072] For those skilled in the art, without departing from the concept of this invention, several simple deductions or substitutions can be made, and all such deductions or substitutions should be considered to fall within the scope of protection of this invention.

Claims

1. An image annotation method for training samples of a deep learning model for fruit and vegetable grading and recognition, characterized in that: Includes the following steps: S1. Acquire images of fruits and vegetables; S2. Preprocess the fruit and vegetable images obtained in step S1. The preprocessing operation includes image segmentation: extracting the fruit and vegetable parts of interest in the image and removing the background parts that are not useful for image recognition. S3. Using the center point of the image as the center, perform fan-shaped segmentation on the image after preprocessing in step S2 to obtain several small block images; each segmented block image has a fruit and vegetable part and a background part, and the fruit and vegetable part is located in the central corner area of ​​the block image, while the background part is located outside the block image. Each segmented block image has the same and uniform characteristics. S4. Perform overall annotation on the segmented images from step S3, where: Cut images of fruits and vegetables without defects and without black spots are labeled as Category I; Images of cut pieces of fruits and vegetables with defects are labeled as Category II; Images of cut fruits and vegetables without defects but with black spots are labeled as Category III; An image with one or more grayscale patches is considered an image with black spots; conversely, an image without any grayscale patches is considered an image without black spots. An image with any of the following fruit and vegetable defects—scabs, wrinkles, punctures, and bruises—is considered an image with fruit and vegetable defects; conversely, an image without any of these defects is considered an image without fruit and vegetable defects.

2. The image annotation method for training samples of the deep learning model for fruit and vegetable grading recognition according to claim 1, characterized in that: The step of cutting the image after preprocessing in step S2 into small pieces in step S3 is to cut the image after preprocessing in step S2 into pieces in a 256*256 matrix form.

3. The image annotation method for training samples of the deep learning model for fruit and vegetable grading recognition according to claim 1, characterized in that: The specific steps in step S3 to cut the image after preprocessing in step S2 into small pieces are to use Python language and call the OpenCV machine vision library to program and implement the automatic image slicing function.

4. The image annotation method for training samples of the deep learning model for fruit and vegetable grading recognition according to claim 1, characterized in that: The image segmentation method in step S2 is any one of the following: threshold-based segmentation, region-based segmentation, edge-based segmentation, and superpixel segmentation.

5. The image annotation method for training samples of the deep learning model for fruit and vegetable grading recognition according to claim 4, characterized in that: The image segmentation processing method in step S2 is a threshold-based segmentation method, which adopts the H component threshold segmentation method based on the HSV model.

6. The image annotation method for training samples of the deep learning model for fruit and vegetable grading recognition according to claim 1, characterized in that: The preprocessing operations in step S2 also include binarization and noise reduction. Image segmentation, binarization, and noise reduction are performed sequentially.

7. The image annotation method for training samples of the deep learning model for fruit and vegetable grading recognition according to claim 6, characterized in that: Noise reduction is achieved by judging the area size during noise reduction processing.

8. The image annotation method for training samples of the deep learning model for fruit and vegetable grading and recognition according to claim 1, characterized in that: Step S1 involves acquiring fruit and vegetable images from multiple angles using multiple cameras. After acquiring the fruit and vegetable images, the number of images is increased using data augmentation methods, including cropping, flipping, and random cropping.

Images