A method and system for improving the accuracy of fruit and vegetable quality detection based on feedback information
By collecting consumer feedback information to generate incremental learning data, the initial fruit and vegetable appearance defect recognition model is incrementally trained. By using AttnGAN and deep residual networks, the problem of inaccurate fruit and vegetable detection results is solved, and a higher detection accuracy is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ZHEJIANG BOSHI NETWORK TECHNOLOGY CO LTD
- Filing Date
- 2026-02-05
- Publication Date
- 2026-06-05
AI Technical Summary
In existing technologies, the results of fruit and vegetable testing are inaccurate, mainly because computer vision recognition models cannot cover all defects in a limited training set, resulting in defects that have not yet appeared being unidentified.
By collecting consumer feedback information through the Internet, incremental learning data is generated to incrementally train the initial fruit and vegetable appearance defect recognition model. AttnGAN and deep residual networks are used to improve the model's parameter update efficiency and defect recognition range.
It improved the accuracy of fruit and vegetable quality testing, enhanced the model's ability to identify untrained defects, and improved the accuracy of the test results.
Smart Images
Figure CN122157243A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of fruit and vegetable testing technology, specifically to a method and system for improving the accuracy of fruit and vegetable quality testing based on feedback information. Background Technology
[0002] Fruit and vegetable quality inspection uses computer vision to identify defects on fruits and vegetables. However, in existing technologies, computer vision recognition algorithms are usually judged on the convergence of the visual recognition model. Model convergence refers to the loss function gradually stabilizing, the parameter update amplitude decreasing, and the model performance reaching a balance on the training and validation sets. When collecting training and test sets, they are usually trained based on images of defects that have already appeared. However, some defects that have not appeared cannot be listed in the limited training or test sets, resulting in inaccurate fruit and vegetable detection results. Summary of the Invention
[0003] To address the technical problems of inaccurate test results in existing fruit and vegetable testing technologies, this invention provides a method and system for improving the accuracy of fruit and vegetable quality testing based on feedback information.
[0004] The technical solution of the present invention to solve the above-mentioned technical problems is as follows: A method for improving the accuracy of fruit and vegetable quality testing based on feedback information includes the following steps: An initial fruit and vegetable appearance defect recognition model was constructed based on computer vision recognition methods. Collect consumer feedback information via the internet; Incremental learning data is generated based on the feedback information; The initial fruit and vegetable appearance defect recognition model is incrementally trained using the incremental learning data to obtain an incremental fruit and vegetable appearance defect recognition model. Collect images of the appearance of fruits and vegetables; The fruit and vegetable appearance defect recognition model is used to identify defects in the fruit and vegetable appearance images to detect the quality of the fruits and vegetables.
[0005] The beneficial effects of this invention are as follows: Since the initial model is trained using a limited number of defect samples, it is impossible for the model to cover all defects. Therefore, it is necessary to continuously update and iterate the model. This invention generates defect sample images of fruits and vegetables by collecting customer feedback information, and then uses the defect sample images of fruits and vegetables to incrementally train the initial fruit and vegetable appearance defect recognition model, thereby improving the efficiency and cost of model parameter updates and improving the defect recognition range and accuracy.
[0006] Based on the above technical solution, the present invention can be further improved as follows.
[0007] Furthermore, the feedback information includes text feedback information and / or image feedback information and / or video feedback information.
[0008] Furthermore, when the feedback information only includes text feedback information, incremental learning data is generated based on the feedback information, including the following steps: Extract text from the text feedback information that meets the preset real feedback conditions and describes the appearance defects of fruits and vegetables; Incremental learning images are generated using extracted text describing the appearance defects of fruits and vegetables; the incremental learning data includes the incremental learning images.
[0009] Furthermore, the preset true feedback condition is that the total number of words describing the same appearance defect of fruits and vegetables in multiple text feedback messages is greater than a preset threshold, and the multiple text feedback messages are not published by the same consumer.
[0010] Furthermore, incremental learning images are generated using the extracted text describing the appearance defects of fruits and vegetables, including the following steps: Obtain a dataset consisting of text and its corresponding images, and perform preprocessing. A text-to-image network model based on AttnGAN is constructed; wherein the text-to-image network model includes a pre-trained network and a multi-stage generation network, wherein the pre-trained network incorporates a Transformer module; and the multi-stage generation network includes at least a first generator, a second generator, and a third generator. Text features are extracted from the text describing the appearance defects of fruits and vegetables. The text features include word features and sentence features. The sentence features are then conditionally enhanced and merged with random noise before being input into the Transformer module to learn spatial and positional information. The learned spatial and positional information is input into the first generator, which outputs a low-resolution image. The generated low-resolution image is downsampled to obtain a first downsampled feature. The word feature is input into a first global attention module to obtain a first word feature. The first downsampled feature and the first word feature are input together into a convolutional neural network to obtain a first fusion feature. The first fusion feature is input into a second generator to output a medium-resolution image. The medium-resolution image is downsampled to obtain a second downsampled feature. The word feature is input into a second global attention module to obtain a second word feature. The second downsampled feature and the second word feature are input together into the convolutional neural network to obtain a second fusion feature. The second fusion feature is input into the third generator to output a high-resolution image. The high-resolution image is used as the incremental learning image.
[0011] Furthermore, when the feedback information includes at least image feedback information, incremental learning data is generated based on the feedback information. Specifically, the images in the image feedback information are processed to remove image segmentation to segment out images containing only fruits and vegetables, thereby obtaining the incremental learning images.
[0012] Furthermore, when the feedback information only includes video feedback information, incremental learning data is generated based on the feedback information, including the following steps: Extract images containing fruits and vegetables from the video feedback information; The image containing fruits and vegetables is segmented to separate images containing only fruits and vegetables, thus obtaining the incremental learning image.
[0013] Furthermore, an initial fruit and vegetable appearance defect recognition model is constructed based on computer vision recognition methods, including the following steps: Collect images of various types of fruit and vegetable appearance defects; A sample dataset was constructed using sample images of various types of fruit and vegetable appearance defects. Build computer vision models; The computer vision model is trained using the sample dataset to obtain the initial fruit and vegetable appearance defect recognition model.
[0014] Furthermore, the initial fruit and vegetable appearance defect recognition model is a deep residual network, which sequentially includes an initial convolutional layer, a max pooling layer, multiple sequentially connected residual blocks, and a global average pooling layer.
[0015] To address the aforementioned technical problems, this invention also provides a system for improving the accuracy of fruit and vegetable quality testing based on feedback information, the specific technical content of which is as follows: A system for improving the accuracy of fruit and vegetable quality testing based on feedback information includes: The model building module is used to build an initial fruit and vegetable appearance defect recognition model based on computer vision recognition methods. The data acquisition module is used to collect consumer feedback information via the Internet; The data generation module is used to generate incremental learning data based on the feedback information; The incremental training module is used to incrementally train the initial fruit and vegetable appearance defect recognition model using the incremental learning data to obtain an incremental fruit and vegetable appearance defect recognition model. The detection module is used to acquire images of the appearance of fruits and vegetables; and to use the fruit and vegetable appearance defect recognition model to identify defects in the fruit and vegetable appearance images in order to detect the quality of the fruits and vegetables. Attached Figure Description
[0016] Figure 1This is a flowchart of a method for improving the accuracy of fruit and vegetable quality detection based on feedback information in an embodiment of the present invention; Figure 2 This is a schematic diagram of a system for improving the accuracy of fruit and vegetable quality detection based on feedback information, as described in an embodiment of the present invention. Detailed Implementation
[0017] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.
[0018] like Figure 1 As shown in the figure, this embodiment provides a method for improving the accuracy of fruit and vegetable quality detection based on feedback information, including the following steps: S1. Construct an initial fruit and vegetable appearance defect recognition model based on computer vision recognition methods; The initial fruit and vegetable appearance defect recognition model is constructed based on computer vision recognition methods, including the following steps: Collect images of various types of fruit and vegetable appearance defects; including at least images of fruit peel damage and fruit peel with dull color.
[0019] A sample dataset was constructed using sample images of various types of fruit and vegetable appearance defects. Build computer vision models; The computer vision model is trained using the sample dataset to obtain the initial fruit and vegetable appearance defect recognition model.
[0020] The initial fruit and vegetable appearance defect recognition model is a deep residual network, which sequentially comprises an initial convolutional layer, a max-pooling layer, multiple sequentially connected residual blocks, and a global average pooling layer. The initial convolutional layer extracts low-level features (such as edges or textures), followed by a max-pooling layer for downsampling. The network constructs a deep architecture by stacking multiple residual blocks, with the output of each block serving as the input to the next. At the end of the network, a global average pooling layer is typically used instead of a traditional fully connected layer to reduce the number of parameters and preserve the spatial information of the feature maps. Finally, a fully connected layer is added for classification or regression tasks.
[0021] Deep Residual Networks (ResNet) effectively alleviate the vanishing and exploding gradient problems in deep networks through residual learning and skip connections. Its training scheme requires a combination of architectural design, optimization strategies, and practical techniques. The core of ResNet is the residual block, whose design goal is to learn the residual mapping (F(x) = H(x) - x) rather than directly fitting the target mapping (H(x)), with the final output being (H(x) = F(x) + x). Skip connections allow gradients to pass directly through. When the residual part (F(x)) is difficult to optimize, the network can maintain performance by setting (F(x) ≈ 0), avoiding degradation caused by increasing the number of layers. For example, batch normalization is introduced after the convolutional layer to reduce internal covariate shifts and stabilize gradient propagation by normalizing the input (mean 0, variance 1). The training process and implementation details, using PyTorch as an example, include data preprocessing, model definition, loss calculation, and optimization iteration. When loading data, the input needs to be normalized (e.g., the mean and standard deviation from ImageNet), and data augmentation (e.g., random cropping, flipping) should be used to improve generalization ability. In the model definition, the initial convolutional layer is followed by multiple residual blocks, each containing a convolutional layer, ReLU activation, and skip connections. The loss function is typically cross-entropy, and the optimizer is recommended to be Adam or SGD (motivated). The initial learning rate can be set to 0.001 and dynamically adjusted using a learning rate scheduler (e.g., cosine annealing). Initialization and normalization: weight initialization uses the Kaiming normal distribution (suitable for ReLU activation) to avoid gradient vanishing; batch normalization further stabilizes training; regularization and acceleration are achieved by adding Dropout (probability 0.3-0.5) to prevent overfitting; weight decay (L2 regularization) is used to constrain model complexity. Gradient control: gradient clipping (threshold 1.0) can alleviate gradient explosion; mixed precision training (AMP) reduces memory usage and accelerates convergence.
[0022] S2. Collect consumer feedback information through the Internet; By using information feedback platforms, consumer feedback can be collected. Additionally, internet search engines can be used to narrow down customer feedback on specific fruits and vegetables. For example, searching for "durian" on an internet platform or search engine, ideally focusing on negative reviews, can yield the aforementioned consumer feedback. To improve the accuracy of feedback, it can be collected from reviews by multiple consumers across multiple platforms. This feedback includes text and / or image and / or video feedback.
[0023] S3. Create incremental learning data based on the feedback information; When the feedback information only includes text feedback information, incremental learning data is generated based on the feedback information, including the following steps: Extract text from the text feedback information that meets the preset real feedback conditions and describes the appearance defects of fruits and vegetables; Incremental learning images are generated using extracted text describing the appearance defects of fruits and vegetables; the incremental learning data includes the incremental learning images.
[0024] The preset true feedback condition is that the total number of words describing the same appearance defect of fruits and vegetables in multiple text feedback messages is greater than a preset threshold, and the multiple text feedback messages are not published by the same consumer.
[0025] Incremental learning images are generated using extracted text describing the appearance defects of fruits and vegetables, including the following steps: Obtain a dataset consisting of text and its corresponding images, and perform preprocessing. A text-to-image network model based on AttnGAN is constructed; wherein the text-to-image network model includes a pre-trained network and a multi-stage generation network, wherein the pre-trained network incorporates a Transformer module; and the multi-stage generation network includes at least a first generator, a second generator, and a third generator. Text features are extracted from the text describing the appearance defects of fruits and vegetables. The text features include word features and sentence features. The sentence features are then conditionally enhanced and merged with random noise before being input into the Transformer module to learn spatial and positional information. The learned spatial and positional information is input into the first generator, which outputs a low-resolution image. The generated low-resolution image is downsampled to obtain a first downsampled feature. The word feature is input into a first global attention module to obtain a first word feature. The first downsampled feature and the first word feature are input together into a convolutional neural network to obtain a first fusion feature. The first fusion feature is input into a second generator to output a medium-resolution image. The medium-resolution image is downsampled to obtain a second downsampled feature. The word feature is input into a second global attention module to obtain a second word feature. The second downsampled feature and the second word feature are input together into the convolutional neural network to obtain a second fusion feature. The second fusion feature is input into the third generator to output a high-resolution image. The high-resolution image is used as the incremental learning image.
[0026] The Transformer module includes the Encoder module and the Decoder module; The Encoder module includes three sequentially connected first sub-modules, each of which includes a self-attention layer, a normalization layer, and a fully connected layer connected in sequence. The Decoder module includes three sequentially connected second sub-modules, each of which includes a first self-attention layer, a first normalization layer, a second self-attention layer, a second normalization layer, and a fully connected layer, all connected in sequence. The output of the Encoder module is respectively input into the second self-attention layer of the three second sub-modules; The first submodule of the Encoder module and the first submodule of the Decoder module correspond to the input terminals, respectively, and the output terminal of the third submodule of the Decoder module corresponds to the output terminal. During the training phase of the AttnGAN-based text-to-image network model, the result of word features passing through the DAMSM module is compared with the result of a 256*256 image passing through an image decoder, and the text-to-image network model is adjusted based on the comparison result.
[0027] The Transformer module is a self-attention-based neural network. The text encoder extracts text features, the image encoder extracts image features, and the DAMSM module inputs the final synthesized image into the image encoder to obtain local image features and text features for correlation comparison, thereby improving the correlation between the image and text. A standard GAN network consists of a generator network (also known as a generator) and a discriminator network (also known as a discriminator). The text-to-image network model of this invention uses three generators and three discriminators, forming three groups. The generators and discriminators are all convolutional neural networks (CNNs).
[0028] The sentence features, after conditional enhancement and merging with random noise, result in a feature vector (collectively referred to as A), which is then input into the Transformer module. In the Encoder module, A is flattened into a one-dimensional vector, and positional information is embedded to obtain the corresponding Q, K, and V matrices, which are then fed into the self-attention layer. In the self-attention layer, weights are calculated for matrices Q and K, and the resulting scores are added to the V matrix. After passing through the self-attention layer, normalization is performed, and the vector is then sent to the fully connected layer, which outputs a one-dimensional vector. This process is repeated three times, and the vector output by the Encoder module becomes B. In the Decoder module, A is flattened into a one-dimensional vector, and positional information is embedded to obtain the corresponding Q, K, and V matrices, which are then fed into the self-attention layer. After passing through the self-attention layer, normalization is performed, and the vector is added to vector B. This vector then enters another self-attention layer to obtain vector C, which is then sent to the fully connected layer. The vector output by the Decoder is the final vector output by the Transformer module.
[0029] When the feedback information includes at least image feedback information, incremental learning data is generated based on the feedback information. Specifically, the images in the image feedback information are processed to remove image segmentation to segment out images containing only fruits and vegetables, thereby obtaining the incremental learning images.
[0030] When the feedback information only includes video feedback information, incremental learning data is generated based on the feedback information, including the following steps: Extract images containing fruits and vegetables from the video feedback information; The image containing fruits and vegetables is segmented to separate images containing only fruits and vegetables, thus obtaining the incremental learning image.
[0031] S4. Use the incremental learning data to incrementally train the initial fruit and vegetable appearance defect recognition model to obtain an incremental fruit and vegetable appearance defect recognition model. S5. Collect images of the appearance of fruits and vegetables; S6. Use the fruit and vegetable appearance defect recognition model to identify defects in the fruit and vegetable appearance images to detect the quality of the fruits and vegetables.
[0032] Since the initial model is trained using a limited number of defect samples, it cannot cover all defects. Therefore, the model needs to be continuously updated and iterated. This invention generates defect sample images of fruits and vegetables by collecting customer feedback information, and then uses these defect sample images to incrementally train the initial fruit and vegetable appearance defect recognition model, thereby improving the efficiency and cost of model parameter updates and improving the defect recognition range and accuracy.
[0033] like Figure 2 As shown, in some other embodiments, a system for improving the accuracy of fruit and vegetable quality detection based on feedback information is also provided, including: The model building module is used to build an initial fruit and vegetable appearance defect recognition model based on computer vision recognition methods. The data acquisition module is used to collect consumer feedback information via the Internet; The data generation module is used to generate incremental learning data based on the feedback information; The incremental training module is used to incrementally train the initial fruit and vegetable appearance defect recognition model using the incremental learning data to obtain an incremental fruit and vegetable appearance defect recognition model. The detection module is used to acquire images of the appearance of fruits and vegetables; and to use the fruit and vegetable appearance defect recognition model to identify defects in the fruit and vegetable appearance images in order to detect the quality of the fruits and vegetables.
[0034] The model building module, data acquisition module, data generation module, incremental training module, and detection module can be program modules, computers, or communication devices or information processing devices with certain computing power.
[0035] In other embodiments, a storage medium is also provided, which stores a computer program or computer instructions. When the computer program or computer instructions are executed by a computer processor, the steps of the above-described method for improving the accuracy of fruit and vegetable quality detection based on feedback information are implemented.
[0036] The storage medium can be an internal storage unit of any data processing device described in any of the foregoing embodiments, such as a hard disk or memory. The storage medium can also be an external storage device of any data processing device, such as a plug-in hard disk, smart memory card, SD card, flash memory card, etc., mounted on the device. Furthermore, the storage medium can include both internal storage units and external storage devices of any data processing device. The computer-readable storage medium is used to store the computer program and other programs and data required by the data processing device, and can also be used to temporarily store data that has been output or will be output.
[0037] In other embodiments, a computer is also provided, including a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they implement the steps of the method for improving the accuracy of fruit and vegetable quality detection based on feedback information in Embodiment 1.
[0038] The memory can be an internal storage unit of any data processing device described in any of the foregoing embodiments, such as a hard disk or RAM. The memory can also be an external storage device of any data processing device, such as a plug-in hard disk, smart memory card, SD card, flash memory card, etc., mounted on the device. Furthermore, the memory can include both internal storage units and external storage devices of any data processing device. The memory is used to store the computer program and other programs and data required by the data processing device, and can also be used to temporarily store data that has been output or will be output.
[0039] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the concept and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for improving the accuracy of fruit and vegetable quality testing based on feedback information, characterized in that, Includes the following steps: An initial fruit and vegetable appearance defect recognition model was constructed based on computer vision recognition methods. Collect consumer feedback information via the internet; Incremental learning data is generated based on the feedback information; The initial fruit and vegetable appearance defect recognition model is incrementally trained using the incremental learning data to obtain an incremental fruit and vegetable appearance defect recognition model. Collect images of the appearance of fruits and vegetables; The fruit and vegetable appearance defect recognition model is used to identify defects in the fruit and vegetable appearance images to detect the quality of the fruits and vegetables.
2. The method for improving the accuracy of fruit and vegetable quality detection based on feedback information according to claim 1, characterized in that, The feedback information includes text feedback information and / or image feedback information and / or video feedback information.
3. The method for improving the accuracy of fruit and vegetable quality detection based on feedback information according to claim 2, characterized in that, When the feedback information only includes text feedback information, incremental learning data is generated based on the feedback information, including the following steps: Extract text from the text feedback information that meets the preset real feedback conditions and describes the appearance defects of fruits and vegetables; Incremental learning images are generated using extracted text describing the appearance defects of fruits and vegetables; the incremental learning data includes the incremental learning images.
4. The method for improving the accuracy of fruit and vegetable quality detection based on feedback information according to claim 3, characterized in that, The preset true feedback condition is that the total number of words describing the same appearance defect of fruits and vegetables in multiple text feedback messages is greater than a preset threshold, and the multiple text feedback messages are not published by the same consumer.
5. The method for improving the accuracy of fruit and vegetable quality detection based on feedback information according to claim 3, characterized in that, Incremental learning images are generated using extracted text describing the appearance defects of fruits and vegetables, including the following steps: Obtain a dataset consisting of text and its corresponding images, and perform preprocessing. A text-to-image network model based on AttnGAN is constructed; wherein the text-to-image network model includes a pre-trained network and a multi-stage generation network, wherein the pre-trained network incorporates a Transformer module; and the multi-stage generation network includes at least a first generator, a second generator, and a third generator. Text features are extracted from the text describing the appearance defects of fruits and vegetables. The text features include word features and sentence features. The sentence features are then conditionally enhanced and merged with random noise before being input into the Transformer module to learn spatial and positional information. The learned spatial and positional information is input into the first generator, which outputs a low-resolution image. The generated low-resolution image is downsampled to obtain a first downsampled feature. The word feature is input into a first global attention module to obtain a first word feature. The first downsampled feature and the first word feature are input together into a convolutional neural network to obtain a first fusion feature. The first fusion feature is input into a second generator to output a medium-resolution image. The medium-resolution image is downsampled to obtain a second downsampled feature. The word feature is input into a second global attention module to obtain a second word feature. The second downsampled feature and the second word feature are input together into the convolutional neural network to obtain a second fusion feature. The second fusion feature is input into the third generator to output a high-resolution image. The high-resolution image is used as the incremental learning image.
6. The method for improving the accuracy of fruit and vegetable quality detection based on feedback information according to claim 3, characterized in that, When the feedback information includes at least image feedback information, incremental learning data is generated based on the feedback information. Specifically, the images in the image feedback information are processed to remove image segmentation to segment out images containing only fruits and vegetables, thereby obtaining the incremental learning images.
7. The method for improving the accuracy of fruit and vegetable quality detection based on feedback information according to claim 3, characterized in that, When the feedback information only includes video feedback information, incremental learning data is generated based on the feedback information, including the following steps: Extract images containing fruits and vegetables from the video feedback information; The image containing fruits and vegetables is segmented to separate images containing only fruits and vegetables, thus obtaining the incremental learning image.
8. The method for improving the accuracy of fruit and vegetable quality detection based on feedback information according to claim 1, characterized in that, The initial fruit and vegetable appearance defect recognition model is constructed based on computer vision recognition methods, including the following steps: Collect images of various types of fruit and vegetable appearance defects; A sample dataset was constructed using sample images of various types of fruit and vegetable appearance defects. Build computer vision models; The computer vision model is trained using the sample dataset to obtain the initial fruit and vegetable appearance defect recognition model.
9. The method for improving the accuracy of fruit and vegetable quality detection based on feedback information according to claim 8, characterized in that, The initial fruit and vegetable appearance defect recognition model is a deep residual network, which includes an initial convolutional layer, a max pooling layer, multiple sequentially connected residual blocks, and a global average pooling layer.
10. A system employing the method for improving the accuracy of fruit and vegetable quality detection based on feedback information as described in any one of claims 1 to 9, characterized in that, include: The model building module is used to build an initial fruit and vegetable appearance defect recognition model based on computer vision recognition methods. The data acquisition module is used to collect consumer feedback information via the Internet; The data generation module is used to generate incremental learning data based on the feedback information; The incremental training module is used to incrementally train the initial fruit and vegetable appearance defect recognition model using the incremental learning data to obtain an incremental fruit and vegetable appearance defect recognition model. The detection module is used to acquire images of the appearance of fruits and vegetables; and to use the fruit and vegetable appearance defect recognition model to identify defects in the fruit and vegetable appearance images in order to detect the quality of the fruits and vegetables.