A large-scale fruit and vegetable image classification method based on machine learning

Abstract

This invention provides a large-scale fruit and vegetable image classification method based on machine learning, belonging to the field of fruit and vegetable classification technology. By using weight range filtering as the priority verification of the visual candidate set, this invention can reduce the misclassification rate of stacked fruits and substandard fruits, while avoiding misclassification of normal large fruits, reducing the direct economic losses caused by incorrect pricing, and ensuring the quality control requirements of supermarkets. For ambiguous scenarios with overlapping weight ranges, through the step-by-step subdivision of HSV hue features and texture features, similar categories can be distinguished without manual intervention, while retaining complete data for subsequent error correction and reducing supermarket operating costs.

Description

Technical Field

[0001] This invention relates to the field of fruit and vegetable classification technology, specifically to a large-scale fruit and vegetable image classification method based on machine learning. Background Technology

[0002] Existing fruit and vegetable image classification technologies are mainly based on deep learning visual models, which extract color, texture, and morphological features from images of fruit and vegetable surfaces to achieve category identification. Visual recognition and weight verification are independent processes. Weight data is used for pricing. For categories with similar weight characteristics, differentiation is usually achieved through manual verification or by supplementing with other modal data such as near-infrared spectroscopy. Data retention relies on subsequent manual sorting.

[0003] In order to grade and price fruits and vegetables according to their size, it is necessary to analyze the corresponding size of fruits and vegetables. However, when two fruits and vegetables are stacked, they are easily misjudged by the visual model as a single qualified fruit. Small, underdeveloped fruits are easily mistakenly classified into qualified grade. At the same time, different categories of fruits and vegetables with similar weight ranges are also prone to identification confusion.

[0004] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to provide a large-scale fruit and vegetable image classification method based on machine learning to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: A large-scale fruit and vegetable image classification method based on machine learning, the specific steps of which include: S1. Place the fruits and vegetables to be classified and weighed on the weighing platform. After the weighing signal is triggered, collect images of the fruits and vegetables in the effective area of ​​the weighing platform. S2. Based on the object detection network, perform object detection and classification reasoning on fruit and vegetable images to generate a candidate category set and corresponding confidence scores; S3. Call the weight range threshold and unit price information for each category of fruits and vegetables; S4. Based on the candidate category set and weight range threshold, filter the current weighing value to determine the unique target category. If multiple candidate categories still exist after filtering, perform secondary subdivision based on the visual features of the fruit and vegetable images to determine the unique target category. S5. Calculate and output the transaction amount based on the unique target product category and its corresponding unit price information; S6. Link and store the image features, weighing values ​​and verification results during the transaction process as audit records.

[0007] Furthermore, during weighing in S1, when the real-time weight value output by the weighing sensor of the weighing platform is greater than the preset trigger threshold and remains stable for more than the first set duration, the image acquisition device captures the current frame image; and crops the current frame based on the platform boundary to obtain an image of fruits and vegetables that only includes the weighing pan bearing surface.

[0008] Furthermore, the cropped image is input into a lightweight object detection network, which outputs bounding box coordinates and classification posterior probability distribution. The top three categories corresponding to the maximum values ​​in the posterior probability distribution are selected as candidate sets, and the maximum probability value is extracted as the confidence level.

[0009] Furthermore, the weight interval thresholds corresponding to each category are obtained through the following method: collecting weight data of samples from each category, calculating the sample weight mean and sample weight standard deviation, and calculating the weight interval thresholds based on the sample weight mean and sample weight standard deviation.

[0010] in, is the mean sample weight, used to reflect the weight trend of this category; k is the confidence coefficient, used to reflect the probability of interval coverage. The standard deviation of sample weight is used to reflect the degree of weight dispersion. The lower limit of permissible weight is the minimum physical boundary value that reflects the weight of this product category. The upper limit of allowable weight is the maximum physical boundary value used to reflect the weight of this product category.

[0011] Furthermore, based on the candidate category set and weight range threshold, the current weighing value is subjected to compatibility filtering, including: obtaining the currently valid weighing value, traversing the candidate category set, extracting the lower and upper limits of the allowable weight corresponding to each candidate category, determining whether the weighing value falls within the allowable weight range of any candidate category, if only one candidate category meets the condition, then the candidate category is determined as the unique target category, if multiple candidate categories meet the condition, then the weight ranges are determined to overlap, and a second segmentation step is triggered.

[0012] Furthermore, the secondary subdivision step includes: extracting the color distribution features of the target region in the fruit and vegetable image, wherein the color distribution features are the histogram distribution of the hue components in the HSV color space, comparing the color distribution features with the standard color features corresponding to each overlapping candidate category, and selecting the candidate category with the highest similarity as the unique target category.

[0013] Furthermore, the secondary segmentation step also includes: if the similarity difference of the color distribution features is less than a preset threshold, then extract the texture complexity features of the fruit and vegetable images, perform a secondary comparison based on the texture complexity features and the standard texture features corresponding to each overlapping candidate category, and select the candidate category with the highest similarity as the unique target category.

[0014] Furthermore, the unit price parameter corresponding to the unique target product category is read. The unit price parameter is the currency value corresponding to each unit weight. The weight value is multiplied by the unit price parameter to obtain the transaction amount.

[0015] Furthermore, the image features, weighing values, and verification results during the transaction process are associated and stored as audit records, generating log entries containing timestamps, image hash values, candidate category sets, weighing values, unique target categories, and transaction amounts, and writing the log entries to the storage medium.

[0016] Compared with the prior art, the beneficial effects of the present invention are: This invention reduces the misjudgment rate of stacked fruits and substandard fruits by using weight range filtering as the priority verification of the visual candidate set, while avoiding misjudging normal large fruits, reducing direct economic losses caused by incorrect pricing, and ensuring the quality control requirements of supermarkets. For ambiguous scenarios with overlapping weight ranges, it can distinguish similar categories without manual intervention by progressively subdividing HSV hue and texture features, while retaining complete data for subsequent error correction and reducing supermarket operating costs. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the overall method flow of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0019] Example: Please see Figure 1 The present invention provides a technical solution: A large-scale fruit and vegetable image classification method based on machine learning, the specific steps of which include: S1. Place the fruits and vegetables to be classified and weighed on the weighing platform. After the weighing signal is triggered, collect images of the fruits and vegetables in the effective area of ​​the weighing platform. During weighing in S1, when the real-time weight value output by the weighing sensor on the weighing platform exceeds the preset trigger threshold and remains stable for more than a first set duration, the image acquisition device captures the current frame image. This eliminates false triggers caused by unstable conditions such as an empty scale, wind, or brief hand contact, ensuring that the objects of subsequent image acquisition are static and complete individual fruits and vegetables. The current frame is then cropped based on the platform boundary to obtain an image of fruits and vegetables containing only the weighing pan's bearing surface. Since the relative position of the camera and the weighing platform is fixed after installation, and the position of the weighing pan's boundary in the image coordinate system is also fixed, cropping transforms the image from a full-scene view to a region of interest view, reducing the difficulty and computational load of subsequent target detection. Complex backgrounds unrelated to fruit and vegetable recognition are removed from the image, allowing computational resources to be concentrated on the fruit and vegetable targets themselves, and eliminating interference from background features.

[0020] S2. Based on the object detection network, perform object detection and classification reasoning on fruit and vegetable images to generate a candidate category set and corresponding confidence scores; The cropped image is input into a lightweight object detection network, which outputs bounding box coordinates and classification posterior probability distribution. The top three categories corresponding to the maximum values ​​in the posterior probability distribution are selected as candidate sets, and the maximum probability value is extracted as the confidence level.

[0021] This lightweight object detection network uses GhostNet as its backbone and embeds an ECA attention module within the feature pyramid network. It automatically extracts color, texture, shape, and contour features of fruits and vegetables using a deep learning model. Through sliding computation with multiple convolutional kernels, it abstracts image features layer by layer. Shallow layers capture low-level features such as edges and color spots, while deep layers combine these low-level features to form high-level semantic features such as the location and overall contour of the fruits and vegetables. The GhostNet structure generates similar feature maps through linear transformations, significantly reducing the number of parameters and computational cost while maintaining feature extraction capabilities. The ECA module adaptively assigns weights to feature maps from different channels using one-dimensional convolutions, allowing the model to focus on the key features of the fruits and vegetables themselves and suppress background noise.

[0022] The head of the object detection network contains two branches: one is responsible for bounding box coordinate regression, which learns the offset between the predicted box and the ground truth box through the loss function; the other is responsible for classification, which outputs the logits of each category through a fully connected layer, and then normalizes them into a probability distribution through the Softmax function. The bounding box coordinates define the region of interest for subsequent color feature extraction to prevent background pixels from contaminating the features. The posterior probability distribution of the classification quantifies the confidence level of each category.

[0023] The higher the probability of a category, the greater the likelihood that it is the true category. Selecting the top three categories is to retain candidate categories whose probability is slightly lower than the highest value but are still competitive, and to further subdivide them using the information.

[0024] S3. Retrieve the weight range thresholds and unit price information corresponding to each category of fruits and vegetables; the stored weight range thresholds corresponding to each category are obtained in the following way: collect the weight data of each category of samples, calculate the sample weight mean and sample weight standard deviation, and calculate the weight range threshold based on the sample weight mean and sample weight standard deviation:

[0025] in, is the mean sample weight, used to reflect the weight trend of this category; k is the confidence coefficient, used to reflect the probability of control interval coverage. The standard deviation of sample weight is used to reflect the degree of weight dispersion. The lower limit of permissible weight is the minimum physical boundary value that reflects the weight of this product category. This is the maximum allowable weight limit, reflecting the maximum physical boundary of the weight for this product category. Abnormal products that are too light or too heavy due to various reasons are filtered out using weight range thresholds.

[0026] During the deployment or category update phase, for each type of fruit and vegetable to be listed, weight data of no fewer than 30 representative individual samples are collected. The statistical characteristics of the samples should stably approximate the true characteristics of the population. Through standardized sampling, extreme individual differences are eliminated, while the commonalities of the category are preserved. The mean and standard deviation of the sample weights are calculated, and based on these two values, a weight range threshold is calculated using a preset weight range threshold formula. This generates a weight acceptance window that encompasses the vast majority of normal individual items and excludes extreme outliers. Any weight outside this range is highly unlikely to be a normal individual item in that category.

[0027] S4. Based on the candidate category set and weight range threshold, filter the current weighing value to determine the unique target category. If only one candidate category meets the conditions, it is directly determined as the unique target category and the decision-making process ends. If no candidate category meets the conditions, a conflict signal is triggered and the process is transferred to manual review. If multiple candidate categories still exist after filtering, a second subdivision is performed based on the visual features of the fruit and vegetable images to determine the unique target category. Based on the candidate category set and weight range threshold, the current weighing value is subjected to compatibility filtering, including: obtaining the currently valid weighing value, traversing the candidate category set, extracting the lower and upper limits of the allowable weight for each candidate category, determining whether the weighing value falls within the allowable weight range of any candidate category, if only one candidate category meets the condition, then the candidate category is determined as the unique target category, if multiple candidate categories meet the condition, then the weight ranges overlap and a secondary segmentation step is triggered.

[0028] The secondary subdivision step includes: extracting the color distribution features of the target region in the fruit and vegetable image, wherein the color distribution features are the histogram distribution of the hue components in the HSV color space, comparing the color distribution features with the standard color features corresponding to each overlapping candidate category, and selecting the candidate category with the highest similarity as the unique target category.

[0029] The pigment composition of different fruit and vegetable varieties exhibits genetic stability. The H hue component in the HSV color space is insensitive to changes in light intensity but sensitive to color differences. Histogram comparisons can quantify the degree of similarity in color distribution between the current sample and the standard. Utilizing the variety-specificity of pigments in fruit and vegetable peels can avoid misclassification of fruits and vegetables of similar weight but different varieties.

[0030] The secondary segmentation step further includes: if the similarity difference of color distribution features is less than a preset threshold, then extracting the texture complexity features of the fruit and vegetable images, such as Local Binary Pattern (LBP) or Histogram of Gradients (HOG). Texture reflects the microscopic geometric structure of the object's surface; even if the colors are the same, there are measurable differences in texture features such as the smoothness of the peel, the presence of fruit spots, and the density of fruit spots. A secondary comparison is then performed between the texture complexity features and the standard texture features corresponding to each overlapping candidate category, selecting the candidate category with the highest similarity as the unique target category.

[0031] S5. Calculate and output the transaction amount based on the unique target product category and its corresponding unit price information; Read the unit price parameter corresponding to the unique target product category. The unit price parameter is the currency value corresponding to each unit weight. Multiply the weighing value by the unit price parameter to obtain the transaction amount.

[0032] S6. Link and store the image features, weighing values ​​and verification results during the transaction process as audit records.

[0033] Image features, weighing values, and verification results during the transaction process are associated and stored as audit records. Log entries containing timestamps, image hash values, candidate category sets, weighing values, unique target category, and transaction amount are generated and written to storage media.

[0034] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0035] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, in the form of a computer program product.

[0036] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A large-scale fruit and vegetable image classification method based on machine learning, characterized in that, The specific steps include: S1. Place the fruits and vegetables to be classified and weighed on the weighing platform. After the weighing signal is triggered, collect images of the fruits and vegetables in the effective area of ​​the weighing platform. S2. Based on the object detection network, perform object detection and classification reasoning on fruit and vegetable images to generate a candidate category set and corresponding confidence scores; S3. Call the weight range threshold and unit price information for each category of fruits and vegetables; S4. Based on the candidate category set and weight range threshold, filter the current weighing value to determine the unique target category. If multiple candidate categories still exist after filtering, perform secondary subdivision based on the visual features of the fruit and vegetable images to determine the unique target category. S5. Calculate and output the transaction amount based on the unique target product category and its corresponding unit price information; S6. Link and store the image features, weighing values ​​and verification results during the transaction process as audit records.

2. The large-scale fruit and vegetable image classification method based on machine learning according to claim 1, characterized in that: When weighing in S1, if the real-time weight value output by the weighing sensor on the weighing platform is greater than the preset trigger threshold and remains stable for more than the first set duration, the image acquisition device captures the current frame image; and crops the current frame based on the platform boundary to obtain an image of fruits and vegetables that only includes the weighing pan bearing surface.

3. The large-scale fruit and vegetable image classification method based on machine learning according to claim 1, characterized in that: The cropped image is input into a lightweight object detection network, which outputs bounding box coordinates and classification posterior probability distribution. The top three categories corresponding to the maximum values ​​in the posterior probability distribution are selected as candidate sets, and the maximum probability value is extracted as the confidence level.

4. The large-scale fruit and vegetable image classification method based on machine learning according to claim 1, characterized in that: The weight interval thresholds for each product category are obtained as follows: Weight data of samples from each product category are collected, the mean and standard deviation of the sample weights are calculated, and the weight interval thresholds are calculated based on the mean and standard deviation of the sample weights. ； in, is the mean sample weight, used to reflect the weight trend of this category; k is the confidence coefficient, used to reflect the probability of interval coverage. The standard deviation of sample weight is used to reflect the degree of weight dispersion. The lower limit of permissible weight is the minimum physical boundary value that reflects the weight of this product category. The upper limit of allowable weight is the maximum physical boundary value used to reflect the weight of this product category.

5. The large-scale fruit and vegetable image classification method based on machine learning according to claim 1, characterized in that: Based on the candidate category set and weight range threshold, the current weighing value is subjected to compatibility filtering, including: obtaining the currently valid weighing value, traversing the candidate category set, extracting the lower and upper limits of the allowable weight for each candidate category, determining whether the weighing value falls within the allowable weight range of any candidate category, if only one candidate category meets the condition, then the candidate category is determined as the unique target category, if multiple candidate categories meet the condition, then the weight ranges overlap and a secondary segmentation step is triggered.

6. The large-scale fruit and vegetable image classification method based on machine learning according to claim 5, characterized in that: The secondary subdivision step includes: extracting the color distribution features of the target region in the fruit and vegetable image, wherein the color distribution features are the histogram distribution of the hue components in the HSV color space, comparing the color distribution features with the standard color features corresponding to each overlapping candidate category, and selecting the candidate category with the highest similarity as the unique target category.

7. The large-scale fruit and vegetable image classification method based on machine learning according to claim 6, characterized in that: The secondary subdivision step further includes: if the similarity difference of color distribution features is less than a preset threshold, then extract the texture complexity features of the fruit and vegetable images, perform a secondary comparison based on the texture complexity features and the standard texture features corresponding to each overlapping candidate category, and select the candidate category with the highest similarity as the unique target category.

8. The large-scale fruit and vegetable image classification method based on machine learning according to claim 1, characterized in that: Read the unit price parameter corresponding to the unique target product category. The unit price parameter is the currency value corresponding to each unit weight. Multiply the weighing value by the unit price parameter to obtain the transaction amount.

9. The large-scale fruit and vegetable image classification method based on machine learning according to claim 1, characterized in that: Image features, weighing values, and verification results during the transaction process are associated and stored as audit records. Log entries containing timestamps, image hash values, candidate category sets, weighing values, unique target category, and transaction amount are generated and written to storage media.

Images