FPGA-based fruit grading method and system

Abstract

The application provides a fruit grading method and system based on FPGA, and relates to the field of image recognition technology. The application uses FPGA as a hardware platform and combines computer vision and machine learning algorithms to realize automatic grading and classification of apples. The programmability and parallel computing capability of FPGA enable the system to quickly process a large amount of image data, improving the real-time performance and efficiency of the grading system. At the same time, by using computer vision, image processing technology and machine learning technology, the accuracy and stability of the grading system are improved through the extraction and analysis of apple features. In addition, the grading system has good scalability and can be customized according to requirements, adapting to fruit grading systems of different scales and requirements.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, specifically to a fruit grading method and system based on FPGA. Background Technology

[0002] Traditional fruit grading typically relies on manual labor, which is time-consuming and susceptible to subjective factors, leading to inconsistencies and inaccuracies in the grading results. With the development of computer vision and machine learning technologies, automated fruit grading systems are gaining increasing attention.

[0003] Currently, automated fruit grading systems are primarily based on computer vision, image processing, and machine learning algorithms. Computer vision extracts various fruit features, such as color, shape, size, and texture, which serve as the basis for classification and grading. Machine learning algorithms, on the other hand, can automatically classify and grade fruit features through training and learning on large amounts of data.

[0004] However, existing automated grading systems still have some limitations in terms of real-time performance, accuracy, scalability, and portability. Real-time performance refers to the system's ability to process and analyze large amounts of image data in a short time to adapt to high-speed production line environments. Accuracy refers to the system's ability to accurately identify and classify fruits to ensure product quality and safety. Scalability refers to the system's ability to be customized to meet different scales and requirements of fruit grading systems. Portability refers to the ability to move equipment to meet different production needs in special production environments. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the present invention aims to propose a fruit grading method based on FPGA, comprising:

[0006] Step 1: Acquire images of the fruit to be evaluated;

[0007] Step 2: Based on FPGA, the image of the fruit to be evaluated is recognized to determine the first grade of the fruit;

[0008] Step 2.1: Preprocess the image of the fruit to be evaluated;

[0009] Step 2.2: The preprocessed image is processed by a pre-trained fruit grading model to obtain the first grade of the fruit to be graded. The first grade represents the grade of the contour of the fruit to be graded. The fruit grading model is obtained by training the model based on training samples. The training samples include input samples and output samples. The input samples are fruit images, and the output samples are the contour grades of the fruit images.

[0010] Step 3: Based on the RGB values ​​of each pixel in the image of the fruit to be evaluated, calculate the hue, lightness, and saturation values ​​of each pixel to determine the target color of the fruit to be evaluated. Based on the preset threshold ranges for hue, lightness, and saturation values ​​of the target color, as well as the hue, lightness, and saturation values ​​of each pixel in the image of the fruit to be evaluated, determine whether each pixel in the image of the fruit to be evaluated belongs to the target color. Based on the ratio of the number of pixels in the fruit to be evaluated that belong to the target color to the total number of pixels in the image of the fruit to be evaluated, determine the second grade of the fruit to be evaluated.

[0011] Step 4: In the preset grading table, determine the target grade of the fruit to be graded based on the first grade and the second grade;

[0012] Step 5: Output the image of the fruit to be evaluated, the image processed by the fruit grading model, and the target grade of the fruit to be evaluated;

[0013] In particular, the calculations in steps 2 to 4 are all performed in the FPGA using parallel and pipelined computing methods.

[0014] Optionally, the fruit grading model comprises convolutional layers, activation layers, pooling layers, and fully connected layers. The convolutional layers include C1, C2, C3, and C4. C1 contains 30 convolutional kernels and has 30 channels. C2 contains 30 convolutional kernels and has 30 channels. C3 contains 16 convolutional kernels and has 16 channels. C4 contains 16 convolutional kernels and has 16 channels. The activation layer uses the LeakyReLU function. The pooling layers include S1 and S2. The fully connected layers include a first fully connected layer, a second fully connected layer, and a third fully connected layer. The first fully connected layer contains 120 neurons, the second fully connected layer contains 84 neurons, and the third fully connected layer contains 4 neurons.

[0015] Based on this, step 2.2 specifically includes:

[0016] The preprocessed image is input sequentially into C1, activation layer, C2, activation layer, S1, C3, activation layer, C4, activation layer, S2, first fully connected layer, second fully connected layer, and third fully connected layer to obtain the first grade of the fruit to be evaluated.

[0017] Optionally, step 2.1 specifically includes:

[0018] Step 2.1.1: Calculate the grayscale values ​​of the pixels in the image of the fruit to be evaluated to obtain the grayscale image;

[0019] Step 2.1.2: The grayscale image is processed using a median filtering noise reduction algorithm to obtain the first image. Specifically, a 3x3 sliding window is defined, which slides across the grayscale image. For the grayscale values ​​of the pixels in the 3 rows and 3 columns of the sliding window, the maximum value, the median value, and the minimum value of each row are determined. The minimum of the three maximum values, the median of the three median values, and the maximum of the three minimum values ​​are then obtained. Finally, the minimum of the three maximum values, the median of the three median values, and the minimum of the three minimum values ​​are obtained. The intermediate value between the large and small values ​​is used as the first intermediate value. It is determined whether the pixels in the 2nd row and 2nd column of the sliding window are in the outline position of the fruit to be evaluated. If the pixels in the 2nd row and 2nd column of the sliding window are in the outline position of the fruit to be evaluated, the gray value of the pixels in the 2nd row and 2nd column of the sliding window remains unchanged. If the pixels in the 2nd row and 2nd column of the sliding window are not in the outline position of the fruit to be evaluated, the gray value of the pixels in the 2nd row and 2nd column of the sliding window is replaced with the first intermediate value. The sliding window is moved so that each pixel in the grayscale image undergoes the above operation. After all pixels have undergone the above operation, the first image is obtained.

[0020] Step 2.1.3: The first image is processed using an improved Sobel sharpening algorithm to obtain the second image. Specifically, for the grayscale values ​​of the first image in the 3*3 sliding window, calculations are performed based on 8 preset sets of different weight parameters to obtain a second intermediate value. Each set of weight parameters is a 3*3 matrix. The grayscale values ​​of the pixels in the 2 rows and 2 columns of the sliding window are replaced with the second intermediate value. The sliding window is moved so that each pixel in the first image undergoes the above operation. After all pixels have undergone the above operation, the second image is obtained.

[0021] Step 2.1.4: Binarize the second image to obtain the preprocessed image.

[0022] Optionally, step 2.1.1 specifically includes:

[0023] For each pixel in the image of the fruit to be evaluated, the grayscale value is calculated using the following formula:

[0024] Gray(i,j)=0.299*R(i,j)+0.587*G(i,j)+0.114*B(i,j)

[0025] Where (i,j) are the coordinates of the pixel in the image of the fruit to be evaluated, R(i,j) is the red component of the pixel in the image of the fruit to be evaluated, G(i,j) is the green component of the pixel in the image of the fruit to be evaluated, B(i,j) is the blue component of the pixel in the image of the fruit to be evaluated, and Gray(i,j) is the gray value of the pixel in the image of the fruit to be evaluated.

[0026] Optionally, in step 3, based on the RGB values ​​of each pixel in the image of the fruit to be evaluated, the hue, lightness, and saturation values ​​of each pixel are calculated to determine the target color of the fruit to be evaluated. This specifically includes:

[0027] For each pixel, obtain the maximum and minimum values ​​of the R, G, and B values. Based on these values, calculate the hue, lightness, and saturation values ​​for each pixel using the following formulas:

[0028]

[0029] V = max

[0030] Where H is the hue value of the pixel, S is the saturation value of the pixel, V is the lightness value of the pixel, G is the G value, B is the B value, R is the R value, min is the minimum value among R, G, and B values, and max is the maximum value among R, G, and B values.

[0031] Optionally, the fruit grading model is deployed on FPGA hardware, and the weight parameters are converted from floating-point data to fixed-point data using the following formula:

[0032] Fixed_data=round(float_data*2 s ),s∈Z

[0033] Where Fixed_data is the fixed-point data of the weight parameters, round(x) is the rounding function, s is the scaling factor, Z represents an integer, and float_data is the floating-point data of the weight parameters.

[0034] An FPGA-based fruit grading system for implementing the FPGA-based fruit grading method of claim 1, comprising an image acquisition sensor, an FPGA, and a display;

[0035] The image acquisition sensor is used to acquire images of the fruit to be evaluated and transmit the acquired images of the fruit to be evaluated to the FPGA;

[0036] The FPGA is used to receive images of the fruit to be evaluated, implement the contents of steps 2 to 4 in claim 1, and transmit the image processed by the fruit grading model and the target grade of the fruit to be evaluated to the display.

[0037] The display is used to output and display the image of the fruit to be evaluated, the image processed by the fruit grading model, and the target grade of the fruit to be evaluated.

[0038] Optionally, when the display outputs the image of the fruit to be evaluated, the image processed by the fruit grading model, and the target grade of the fruit to be evaluated, the row and column ranges of the image of the fruit to be evaluated on the display screen are both set to [100, 323], representing from row 100 to row 323 and from column 100 to column 323; the row range of the image processed by the fruit grading model is [0, 223], representing from row 0 to row 223; and the column range is [400, 623], representing from column 400 to column 623; the row range of the target grade of the fruit to be evaluated is [0, 99], representing from row 0 to row 99; and the column range is [100, 199], representing from column 100 to column 199.

[0039] The beneficial effects of adopting the above technical solution are as follows:

[0040] This invention utilizes an FPGA as the hardware platform and combines computer vision and machine learning algorithms to achieve automatic grading and classification of apples. Leveraging the programmability and parallel computing capabilities of the FPGA, the system can rapidly process large amounts of image data, improving the real-time performance and efficiency of the grading system. Simultaneously, by employing computer vision, image processing, and machine learning techniques to extract and analyze apple features, the accuracy and stability of the grading system are enhanced. Furthermore, the grading system of this invention has good scalability and can be customized to meet specific needs, adapting to fruit grading systems of different scales and requirements. Attached Figure Description

[0041] Figure 1 This is a schematic diagram of a fruit grading method based on FPGA in an embodiment of the present invention;

[0042] Figure 2 These are eight sets of weight parameter templates in this embodiment of the invention. Figure a is a template for a weight parameter of 0°, Figure b is a template for a weight parameter of 45°, Figure c is a template for a weight parameter of 90°, Figure d is a template for a weight parameter of 135°, Figure e is a template for a weight parameter of 180°, Figure f is a template for a weight parameter of 225°, Figure g is a template for a weight parameter of 270°, and Figure h is a template for a weight parameter of 315°.

[0043] Figure 3 This is a schematic diagram of the fruit grading model in an embodiment of the present invention;

[0044] Figure 4 This is a schematic diagram of the structure of a fruit grading system based on FPGA in an embodiment of the present invention;

[0045] Figure 5This is a functional structural diagram of a fruit grading system based on FPGA in an embodiment of the present invention. Detailed Implementation

[0046] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0047] To address the problems of existing technologies, this invention provides a fruit grading method and system based on FPGA. The FPGA-based fruit grading method primarily involves software design, while the FPGA-based fruit grading system mainly involves the hardware design and implementation of the software. This invention accurately processes and analyzes fruit images and identifies the grade of apples on the production line according to grading standards. The technical solution is as follows: the FPGA hardware platform interacts with an image acquisition device to capture apples moving on the production line. The acquired images are transmitted to the hardware platform for recognition and processing. First, the image data stream is improved in quality through image processing technology and then divided into two paths. One path uses image processing technology to grade based on color standards, while the other path uses a deep learning algorithm to grade based on fruit shape and defect standards. Finally, the judgment results from both paths are combined to obtain the final grading result, which is then displayed in a specific area on the display device. Figure 1 Specifically, this may include the following steps:

[0048] Step 1: Acquire images of the fruit to be evaluated;

[0049] Step 2: Based on FPGA, the image of the fruit to be evaluated is recognized to determine the first grade of the fruit;

[0050] Step 2.1: Preprocess the image of the fruit to be evaluated;

[0051] Step 2.1.1: Calculate the grayscale values ​​of the pixels in the image of the fruit to be evaluated to obtain the grayscale image.

[0052] Specifically, this invention addresses the limitation of FPGAs in performing decimal calculations and, to avoid resource waste, converts the binary weighting coefficients into integers by shifting them left by 8 bits (equivalent to multiplying by 256). Based on empirical values ​​of human eye sensitivity to R, G, and B components, the obtained weighting parameters are 0.299, 0.587, and 0.114, respectively. Based on this, the grayscale value is calculated for each pixel in the image of the fruit to be evaluated using the following formula:

[0053] Gray(i,j)=0.299*R(i,j)+0.587*G(i,j)+0.114*B(i,j)

[0054] Where (i,j) are the coordinates of the pixel in the image of the fruit to be evaluated, R(i,j) is the red component of the pixel in the image of the fruit to be evaluated, G(i,j) is the green component of the pixel in the image of the fruit to be evaluated, B(i,j) is the blue component of the pixel in the image of the fruit to be evaluated, and Gray(i,j) is the gray value of the pixel in the image of the fruit to be evaluated.

[0055] In the FPGA operation, Gray represents grayscale data, R represents the red component, G represents the green component, and B represents the blue component.

[0056] To avoid timing issues caused by excessive logic, this invention utilizes FPGA pipeline design methods, sacrificing some area for faster processing speed. Specifically, the grayscale process employs a three-stage pipeline design: the first stage handles multiplication, the second stage performs addition, and the third stage performs shift restoration. Since the entire grayscale process requires three clock cycles, the flag indicating valid data needs to be delayed by three cycles to correspond with the data.

[0057] Step 2.1.2: The grayscale image is processed using a median filtering noise reduction algorithm to obtain the first image. Specifically, a 3x3 sliding window is defined, which slides across the grayscale image. For the grayscale values ​​of the pixels in the 3 rows and 3 columns of the sliding window, the maximum value, the median value, and the minimum value of each row are determined. The minimum of the three maximum values, the median of the three median values, and the maximum of the three minimum values ​​are then obtained. Finally, the minimum of the three maximum values, the median of the three median values, and the minimum of the three minimum values ​​are obtained. The intermediate value between the large and small values ​​is used as the first intermediate value. It is determined whether the pixels in the 2nd row and 2nd column of the sliding window are in the outline position of the fruit to be evaluated. If the pixels in the 2nd row and 2nd column of the sliding window are in the outline position of the fruit to be evaluated, the gray value of the pixels in the 2nd row and 2nd column of the sliding window remains unchanged. If the pixels in the 2nd row and 2nd column of the sliding window are not in the outline position of the fruit to be evaluated, the gray value of the pixels in the 2nd row and 2nd column of the sliding window is replaced with the first intermediate value. The sliding window is moved so that each pixel in the grayscale image undergoes the above operation. After all pixels have undergone the above operation, the first image is obtained.

[0058] Specifically, a 3x3 sliding window is defined as follows:

[0059] Generating a 3×3 window requires buffering 3 rows of pixels. During the design process, a first-in-first-out (FIFO) memory was used to buffer two rows of pixels, and the row buffers were connected in a daisy chain, that is, the output of the current row buffer was connected to the input of the next row buffer. The third row of pixels appeared as real-time input.

[0060] It should be noted that for pixels at the boundaries of the grayscale image, such as pixels in the first row, first column, last row, and last column of the grayscale image, when performing the above operation, since these pixels cannot be positioned in the two-row, two-column position of the sliding window, the pixels on the boundary are copied outside the image, and the rows and columns inside the boundary are mirrored outside the image. For example, the pixels in the first row are copied to row 0, so that the pixels in the first row can be positioned in the two-row, two-column position of the sliding window.

[0061] Step 2.1.3: Process the first image using an improved Sobel sharpening algorithm to obtain the second image. Since the energy of the image is mainly concentrated in the low-frequency region, while noise and edge information are more prevalent in the high-frequency region, the high-frequency components are enhanced by performing differentiation operations on the image, making the image clearer. The Sobel sharpening algorithm is an image processing technique based on the Sobel operator, used to enhance the edges and details of an image. The algorithm's idea is to first perform an averaging operation on the image, then perform differentiation operations, and finally calculate the gradient. Through averaging, the low-frequency region of the image can be filtered, effectively suppressing noise in the high-frequency region. The Sobel operator has relatively low sensitivity to noise. The calculation principle of the Sobel sharpening algorithm is explained below.

[0062] For the implementation of the Sobel algorithm, this invention also uses a FIFO data buffer to implement sliding window processing. The 3x3 window only needs two FIFO buffers to buffer the data of the previous row and the row before that, and uses the currently transmitted data as the third row, thereby completing the arrangement of the three rows of data.

[0063] Specifically, for the grayscale value of the first image in the 3*3 sliding window, a second intermediate value is obtained by calculating according to 8 preset sets of different weight parameters. Each set of weight parameters is a 3*3 matrix. The grayscale value of the pixels in the 2 rows and 2 columns of the sliding window is replaced with the second intermediate value. The sliding window is moved so that each pixel in the first image undergoes the above operation. After all pixels have undergone the above operation, the second image is obtained.

[0064] This invention employs an 8-directional convolution factor, i.e., 8 different sets of weight parameters, to achieve more refined edge detection and more complete image features. Based on the original directions, six additional directions—45°, 135°, 180°, 225°, 270°, and 315°—are added, resulting in eight directional templates used as weight parameters. The eight different weight parameter sets are referenced... Figure 2This invention utilizes the parallel computing capabilities of FPGAs to simultaneously perform gradient calculations in eight directions, while requiring the same amount of time as the original Sobel algorithm. This optimizes the algorithm while meeting the requirements of real-time systems.

[0065] The data processed by the image denoising algorithm is first cached in two FIFO memories. A 3x3 window is constructed using a row-by-row delay method, ensuring that each pixel forms a 3x3 window with its eight neighboring pixels. Unlike median filtering, the sharpening filter includes weight parameters, which are used in conjunction with the nine corresponding pixels in the window. These pixels are represented as matrix_p00 to matrix_p22 in the image. Due to the characteristics of sequential logic, the result of each calculation is output in the next cycle. Finally, this paper employs a successive approximation method, which involves continuously comparing data and taking the square root of the squares G_square_data of the calculation results in the horizontal and vertical directions to obtain the final result G_data.

[0066] Step 2.1.4: Binarize the second image to obtain the preprocessed image.

[0067] Step 2.2: The preprocessed image is processed using a pre-trained fruit grading model to obtain the first grade of the fruit to be graded. The first grade represents the grade of the fruit's outline. For example,

[0068] The fruit grading model is obtained by training the model based on training samples, which include input samples and output samples. The input samples are fruit images, and the output samples are the contour grades of the fruit images.

[0069] The contour levels in the output samples are manually labeled, specifically according to the following standards:

[0070] Superior grade, the fruit shape has the characteristics that this product should have, and the fruit shape is free of defects.

[0071] First-class products are allowed to have slight defects in fruit shape, but the fruit shape is not defective.

[0072] Second-class products are allowed to have some defects in fruit shape, but still maintain the basic characteristics of the product. Deformed fruits are not allowed, and the area of ​​damage such as punctures (skin breaks and scratches), hail damage, insect damage, and brown rust is less than 10%.

[0073] Substandard products are those with deformed fruit shapes and damage areas exceeding 10% including punctures (skin tears), hail damage, insect damage, and brown rust.

[0074] In terms of the shape of the fruit, superior, first-class, second-class and unclassified products correspond to the first grade.

[0075] Since the recognition accuracy and structural stability of the Lenet-5 network have been widely verified, and its application scenarios are similar to those of this invention, the fruit grading model described in this invention is an improvement based on the Lenet-5 network, specifically including the following parts:

[0076] (1) Convolutional Layers: The original network structure has two types of convolutional layers: one with six 5x5 kernels and the other with sixteen 5x5 kernels. For the first type of convolutional layer, considering that the features of apple images are more complex than handwritten digits, the number of channels is expanded to 30 to extract features more effectively. Secondly, regarding the size of the convolutional kernels, from the perspective of hardware platform implementation, the 5x5 convolutional kernels are replaced with two 3x3 convolutional kernels. Without changing the receptive field of the convolutional kernels, the computational load is reduced and the feature extraction capability is improved. Under the 224x224 input condition of this invention, the multiplication and addition computation loads can both be reduced by 1.3 times.

[0077] (2) Regarding the activation function: To avoid the vanishing gradient problem, a nonlinear relationship is introduced to handle complex data. This invention chooses to use the LeakyReLU function instead of the Sigmoid function in the original Lenet-5 network. This expands the range of input value mapping from [0,1] to [-∞, +∞]. Most importantly, when deploying the network on a hardware platform, the LeakyReLU function will enhance the performance of signals with values ​​greater than a certain threshold.

[0078] (3) Pooling and Fully Connected Layers: This invention modifies the number of neurons in the output layer of the fully connected layer based on the original model. The final network structure contains four convolutional layers, with C1 and C2 each containing 30 convolutional kernels, and C3 and C4 each containing 16 convolutional kernels. In addition, the model includes two max-pooling layers, S1 and S2, located after the activation layers of the C2 and C4 convolutional layers, respectively. Finally, three fully connected layers are used for classification, containing 120, 84, and 4 neurons, respectively.

[0079] Therefore, the improved model, namely the fruit grading model, consists of convolutional layers, activation layers, pooling layers, and fully connected layers. The convolutional layers include C1, C2, C3, and C4. C1 contains 30 convolutional kernels and has 30 channels. C2 contains 30 convolutional kernels and has 30 channels. C3 contains 16 convolutional kernels and has 16 channels. C4 contains 16 convolutional kernels and has 16 channels. The activation layer uses the LeakyReLU function. The pooling layers include S1 and S2. The fully connected layers include a first fully connected layer, a second fully connected layer, and a third fully connected layer. The first fully connected layer contains 120 neurons, the second fully connected layer contains 84 neurons, and the third fully connected layer contains 4 neurons.

[0080] It should be noted that the training samples for the fruit grading model in this section are for Fuji apples, and the parameters of each layer of the fruit grading model are also for Fuji apples. Therefore, the accuracy of the above parameters in grading Fuji apples is higher. For grading other fruits, it may be necessary to adjust the training samples and retrain.

[0081] Based on this, the fruit grading model was tested, specifically:

[0082] The host computer experimental environment of this invention is conducted in Matlab, including the construction, training, and testing of the neural network. Furthermore, since the final network model needs to be deployed on an FPGA platform, the parameters need to be stored on the hardware platform in advance. After multiple experiments, a set of hyperparameters with relatively good grading performance was finally determined. The determined training parameters include a total number of training epochs of 50; a maximum number of iterations of 6000; a batch size of 64 training samples per batch; a learning rate of 0.0001; and an optimizer of Sgdm. Matlab is needed to convert the trained network parameters according to the requirements of the hardware platform. A self-made apple dataset is used to train and validate the network models before and after the improvement. The grading performance of the improved network model is evaluated by comparing accuracy and loss performance.

[0083] The improved network model achieved a 3% increase in the accuracy of apple image grading, and the validation set loss and training loss of the improved network showed better fit. To further evaluate the performance of the network model, this invention used a confusion matrix for detailed analysis.

[0084] The horizontal axis represents the predicted class of apples, and the vertical axis represents the actual class. The test results are as follows: out of 25 samples, the predicted results for premium apples matched the actual results in 22 cases, with an accuracy rate of 88%; the predicted results for first-class apples matched in 21 cases, with an accuracy rate of 84%; the predicted results for second-class apples matched in 23 cases, with an accuracy rate of 92%; and the predicted results for substandard apples matched in 22 cases, with an accuracy rate of 88%. The average accuracy rate reached 88%, which is basically consistent with the above network validation results.

[0085] Based on this, combined Figure 3 Step 2.2 specifically includes:

[0086] The preprocessed image is input sequentially into C1, activation layer, C2, activation layer, S1, C3, activation layer, C4, activation layer, S2, first fully connected layer, second fully connected layer, and third fully connected layer to obtain the first grade of the fruit to be evaluated.

[0087] Specifically, the preprocessed image is input into C1. A first sliding window is constructed based on the size of the convolution kernel in C1. Based on this first sliding window, convolution operations are performed on the preprocessed image in each channel of C1 according to the channel weights. Then, all images output from C1 are processed by the LeakyReLU activation layer. The images processed by the activation layer are then input into C2. A second sliding window is constructed based on the size of the convolution kernel in C2. Based on this second sliding window, convolution operations are performed on the images processed by the activation layer in each channel of C2 according to the channel weights. Then, all images output from C2 are processed by the LeakyReLU activation layer. The images processed by the activation layer are then input into S1. A 2x2 sliding window is used in S1 to slide through each image processed by the activation layer, selecting the maximum value within the 2x2 sliding window. Finally, all images output from S1 are input into C3. A third sliding window is constructed based on the size of the convolution kernel in C3. A sliding window is constructed, and simultaneously, in each channel of convolutional layer C3, convolution operations are performed on all images output from S1 according to the channel weights. Then, all images output from C3 are processed by the LeakyReLU activation layer. The processed images are then input into C4. Based on the size of the convolution kernel in C4, a fourth sliding window is constructed. Based on this fourth sliding window, simultaneously, in each channel of convolutional layer C4, convolution operations are performed on all images processed by the activation layer according to the channel weights. Then, all images output from C4 are processed by the LeakyReLU activation layer. The function processes the image, which is then input into S2 after being processed by the activation layer function. According to the 2*2 sliding window in S2, the 2*2 sliding window slides through each image processed by the activation layer function, and selects the maximum value in the 2*2 sliding window. Then, all the images output by S2 are passed through 120 neurons in the first fully connected layer to obtain 120 data points. The 120 data points are input to obtain 84 data points. The 84 data points are input to 4 neurons in the third fully connected layer to obtain 4 data points, which gives the first grade of the fruit to be evaluated.

[0088] When implementing convolutional neural networks on a software platform, floating-point arithmetic is primarily used. To ensure the accuracy and stability of network training, 32-bit or 64-bit floating-point precision is typically used to train the weight parameters. However, high-precision floating-point representation means significant memory consumption. Simultaneously, as network depth increases, the number of weight parameters grows exponentially, greatly slowing down model training. When deploying networks on an FPGA hardware platform, using fixed-point integer data offers higher cost-effectiveness. Based on this, the fruit grading model in this invention performs fixed-point quantization on the weight parameters to achieve FPGA hardware deployment. Specifically, the following formula is used to convert the weights and related parameters into fixed-point numbers and store them on the FPGA platform.

[0089] Fixed_data=round(float_data*2 s ),s∈Z

[0090] Where Fixed_data is the fixed-point data of the weight parameters, round(x) is the rounding function, s is the scaling factor, Z represents an integer, and float_data is the floating-point data of the weight parameters.

[0091] Since this invention is implemented on an FPGA, the image processing program and image recognition program in the software part are both written in the hardware description language VHDL Erilog and embedded in the FPGA. The image processing program preprocesses the image data captured by the image acquisition device and performs grading according to color standards. The image recognition program analyzes the preprocessed image and extracts the target fruit shape and defect features. Therefore, when programming the fruit grading model of this invention, further design is needed for the convolutional modules corresponding to the convolutional layers, the pooling modules corresponding to the pooling layers, and the fully connected layer modules corresponding to the fully connected layers. Specifically:

[0092] (1) The specific design and implementation of the convolution module are as follows:

[0093] Traditional software implementations of convolution require multiple loops, which involve extensive parallel computation. Therefore, this invention addresses the computational characteristics of convolutional layers by fully utilizing hardware parallelism, dividing the convolution module into a caching module and a convolution computation module.

[0094] The caching module consists of two parts: a data cache (Data_ram) and a weight cache (Param_ram). It primarily utilizes on-chip memory resources. Besides the memory itself, this module also requires corresponding read / write control programs. After a frame of image data is cached, a completion flag is set high and passed to the convolution calculation module. The convolution calculation module sends a parameter fetch command to Param_ram and a data read command to Data_ram. Since the weight parameters read sequentially are much smaller than the image data, a 3x3 window only requires 3 cycles. The weight parameters need to be held until the image data and weights are perfectly aligned before performing convolution calculations. This process is repeated until all channels have been calculated.

[0095] The convolution computation module is crucial to the entire convolutional layer hardware implementation. First, based on the kernel size, the module reads data from the `Data_ram` using commands including the read address `data_rd_addr` and the read data `row_cnt`. The `Data_ram` receives image data and constructs a window template through appropriate timing control. To retrieve the first three rows of data, a three-cycle delay is set to ensure that the first three elements of a row are processed within the same cycle. This design effectively supports window template construction and parallel data processing. Second, the convolution computation module reads weight parameters from `Param_rom` using the parameter read address `param_rd_addr` and the parameter read command `conv_cnt`.

[0096] (2) The specific implementation of the pooling module is as follows:

[0097] This invention employs 2×2 max pooling, maximizing pixel features by selecting the most significant pixel for output. The pooling module is implemented in three steps. First, it compares data within the same row. The input data processed in the previous layer is counted by row and column. Considering that this paper uses window max pooling, for a single window, when the first row count is odd and the column count is even, it proves that at least two data points have entered the module, meeting the pooling requirements. Second, the raw input data is delayed by one cycle to ensure that the first and second data points in the first row correspond. Next, a FIFO is used as a data buffer, selecting the maximum value of the first and second data points as the input `fifo_wr_data`. The FIFO buffers the maximum values ​​of all pairwise data points in a row, separated by a factor of 2.

[0098] (3) The full-connection module design and implementation are as follows:

[0099] In the network structure, the fully connected layer is located at the end. It is responsible for transforming the original neurons, arranged into a two-dimensional matrix, into a one-dimensional matrix. Due to the large number of weights, to conserve hardware resources, this paper places the fully connected layer in the PS (Processing System) part and designs and deploys it using ARM. All parameters involved in the fully connected layer are stored in DDR3 memory. During computation, the weight parameters at a specified address are read and multiplied / added with the data passed from the PL (Programmable Logic) part to the corresponding address in DDR3, thus completing the computation of the fully connected layer.

[0100] Step 3: Based on the RGB values ​​of each pixel in the image of the fruit to be evaluated, calculate the hue, lightness, and saturation values ​​of each pixel to determine the target color of the fruit to be evaluated. Based on the preset threshold ranges for hue, lightness, and saturation values ​​of the target color, as well as the hue, lightness, and saturation values ​​of each pixel in the image of the fruit to be evaluated, determine whether each pixel in the image of the fruit to be evaluated belongs to the target color. Based on the ratio of the number of pixels in the fruit to be evaluated that belong to the target color to the total number of pixels in the image of the fruit to be evaluated, determine the second grade of the fruit to be evaluated.

[0101] It should be noted that the target color is determined based on the fruit to be evaluated. For example, if the fruit to be evaluated is a Fuji apple, the target color is red; if the fruit to be evaluated is a green mango, the target color is green. Since this invention is for identifying Fuji apples, the target color is red. In the following detailed description, the target color is described as red.

[0102] The RGB color space precisely controls the mixing of the three primary colors—red, green, and blue—by adjusting the intensity of each channel to obtain the desired color. In contrast, the HSV color space is more user-friendly and better reflects human subjective perception of color. This color space is mainly composed of three dimensions: hue (H), saturation (S), and value (V). By adjusting these three dimensions, it is easier to select and control the desired color. This paper determines the chromaticity of red through hue components. Since the hue range of red is 0–10 and 156–180, color feature extraction involves statistically analyzing the pixel percentage within this range and classifying it according to a grading standard. Therefore, HSV is more suitable for grading color conditions in this invention.

[0103] Based on this, step 3 calculates the hue, brightness, and saturation values ​​of each pixel in the image of the fruit to be evaluated, and determines the target color of the fruit to be evaluated, specifically including:

[0104] For each pixel, obtain the maximum and minimum values ​​of the R, G, and B values. Based on these values, calculate the hue, lightness, and saturation values ​​for each pixel using the following formulas:

[0105]

[0106] V = max

[0107] Where H is the hue value of the pixel, S is the saturation value of the pixel, V is the lightness value of the pixel, G is the G value, B is the B value, R is the R value, min is the minimum value among R, G, and B values, and max is the maximum value among R, G, and B values.

[0108] In the FPGA implementation, the color space conversion algorithm prioritizes acquiring the maximum and minimum values ​​of the RGB three-channel components in the first cycle, and first calculates the lightness and saturation to allow sufficient time for the hue (H) calculation. To improve efficiency, combinational logic is used to perform the difference operation between the maximum and minimum values ​​in the first cycle. As shown in the formula above, the difference operation between the RGB three-channel components in the final formula requires multiplication by 60. Therefore, in the first cycle, each extracted RGB three-channel component is multiplied by 60 to maintain consistency with the final result. Since the magnitude relationship of the RGB three-channel components determines different calculation methods, in the second cycle, the magnitude relationship of the three-channel components needs to be determined first, and the value of the largest channel component needs to be identified to select the appropriate difference operation formula and the final calculation formula. Finally, all calculations are completed in the third cycle. In this way, the system can implement the color space conversion algorithm in only three clock cycles, greatly improving the execution speed of the algorithm. After the conversion is completed, the number of pixels is counted based on the range of red pixels.

[0109] After conversion, the standard based on color distribution classifies products into superior, first, second, and ungraded grades. A product is classified as superior if the area covered by red or red stripes is over 88%; first grade if the area covered by red or red stripes is between 78% and 88%; second grade if the area covered by red or red stripes is between 53% and 78%; and ungraded if the area covered by red or red stripes is below 53%. Specifically, for Fuji apples, superior, first, second, and ungraded grades correspond to the second level.

[0110] Step 4: In the preset grading table, determine the target grade of the fruit to be graded based on the first grade and the second grade;

[0111] In practical implementation, this invention can set target levels for different first and second levels. For example, if the first level is 1 and the second level is 1, then the target level is 1; if the first level is 1 and the second level is 3, then the target level is 2. Based on this list, rows represent the four levels of the first level, columns represent the four levels of the second level, and the corresponding target levels, thus obtaining a grading table. Based on this, after obtaining the first and second levels, this invention can search the grading table to determine the target level of the fruit to be evaluated.

[0112] Step 5: Output the image of the fruit to be evaluated, the image processed by the fruit grading model, and the target grade of the fruit to be evaluated.

[0113] In particular, the calculations in steps 2 to 4 are all performed in the FPGA using parallel and pipelined computing methods.

[0114] This invention also provides an FPGA-based fruit grading system for implementing an FPGA-based fruit grading method, combined with... Figure 4 This includes image acquisition sensors, FPGAs, and displays;

[0115] The image acquisition sensor is used to acquire images of the fruit to be evaluated and transmit the acquired images of the fruit to be evaluated to the FPGA;

[0116] In this invention, the image data acquisition is accomplished jointly by hardware and software drivers.

[0117] (1) The hardware driver uses a low-power CMOS image sensor from OmniVision and an adapter board. The CMOS image sensor has 5 megapixels. In the specific implementation, the CMOS image sensor is an OV5640 camera, and the adapter board is a PMOD-5M acquisition component. This invention acquires 640x480 resolution apple images, and the sensor can achieve a maximum frame rate of 90fps.

[0118] (2) The implementation of software-driven systems involves three steps:

[0119] Step 1: This invention includes a clock power-on initialization program that provides clock and reset signals for image acquisition. Specifically:

[0120] In the initialization module, the Zedboard controls the power-down enable (PWDN), reset (RESETB), and system input clock (XVCLK). After the chip pin voltage (DOAVDD) and analog voltage (AVDD) are pulled high for at least 6ms, PWDN is pulled low and held. After PWDN is pulled low for at least 1ms, a low-level reset (RESETB) is pulled high, and RESETB is ensured to be pulled low at least 2ms before AVDD is pulled low.

[0121] Step 2: Implement data transmission within the data acquisition process by writing an I2C transmission protocol. The complete I2C protocol consists of two parts, specifically:

[0122] (1) The I2C write data timing process begins with the master sending a start signal (START), followed by sending the slave's device address to select the slave to communicate with. If the slave device successfully receives the instruction and is ready to communicate, it will send an acknowledgment bit (ACK); otherwise, it will not respond. After receiving the acknowledgment bit, the master sends data to the slave. The master device sequentially sends data to the data line SDA while the clock line SCL is low. Finally, after the slave has finished receiving the data, it sends an acknowledgment bit (ACK) to indicate that it is ready to receive the next byte of data. After writing all the data, the master issues a stop condition to indicate the end of one write operation.

[0123] (2) The entire I2C data reading timing process can be divided into two parts. The first part requires sending a write data command before the master sends the read data command. After the slave responds, the master sends a start signal again, and only then does the data reading process truly begin. Similar to writing data, the master sends the slave device address and read flag command. After the slave responds, the slave device sends data to the SDA line in sequence. If the master does not respond after reading the data, it means that the data receiving process is complete.

[0124] Step 3: Develop an image data acquisition module. Based on the relevant instructions in the registers, sample the image sensor and read the acquired image data row by row and column by column. The output image data is in 16-bit RGB565 format. However, only 8 bits of data can be acquired at a time. Therefore, it is necessary to concatenate two consecutive bytes to obtain the complete 16-bit data. The first byte can be used as the high 8 bits, and the second byte as the low 8 bits; combining these two bytes yields the complete 16-bit RGB565 data.

[0125] The FPGA is used to receive images of the fruit to be evaluated, implement the contents of steps 2 to 4 in claim 1, and transmit the image processed by the fruit grading model and the target grade of the fruit to be evaluated to the display.

[0126] In this embodiment, the FPGA platform is a flat board measuring 6.3 inches × 5.3 inches.

[0127] Combination such as Figure 4 The FPGA includes an image driver module, an image processing module, a convolutional neural network hierarchical module, a system main control module, and an image display driver module.

[0128] The system main control module, as the core control part of the entire system, is mainly used to drive hardware components to achieve corresponding functions through the processor system on the FPGA platform. The image driver module, image processing module, convolutional neural network hierarchical module and image display driver module are all written in the PL part. The entire system is made to run by writing some instructions in the ARM of the PS part. In the specific implementation, the ARM model is Cortex-A9.

[0129] The image driver module is used to send instructions to the image acquisition sensor, which in turn drives the image acquisition sensor to transmit the acquired fruit image to be evaluated. Specifically, the image acquisition sensor transmits the fruit image to be evaluated to the image driver module, which then transmits the fruit image to the image processing module.

[0130] The image processing module is used to preprocess the images of the fruit to be evaluated and to determine the second grade of the fruit to be evaluated.

[0131] The convolutional neural network grading module is used to process the pre-trained fruit grading model to obtain the first grade portion of the fruit to be graded;

[0132] The image display driver module is used to configure some displays and extracts the processed data from the DDR memory controller and transmits it to the display via a protocol.

[0133] The display is used to output and display the image of the fruit to be evaluated, the image processed by the fruit grading model, and the target grade of the fruit to be evaluated. The display is an LCD liquid crystal display with a resolution of 1440×900.

[0134] When outputting an image of the fruit to be evaluated, the image processed by the fruit grading model, and the target grade of the fruit to be evaluated on a display, this invention includes the following design based on the actual use of the display:

[0135] (1) Display path implementation

[0136] Furthermore, the specific steps include the following:

[0137] Furthermore, step (11) specifically includes:

[0138] VGA monitors display images in a progressive scan mode, scanning the pixels that make up the image onto the screen sequentially from top to bottom and from left to right under horizontal and vertical sync signals. A complete horizontal scan cycle includes the horizontal sync pulse signal (HSync), back header, left border, valid video, right border, and front header. At a timing standard of 640x480@60Hz, one horizontal scan cycle includes 800 pixels, of which 96 are for the sync signal, 40 for the back header, 8 for the left border, 640 for the valid video, 8 for the right border, and 8 for the front header. The parameters and processes of the vertical scan cycle differ from those of the horizontal scan cycle. The vertical scan cycle is defined as the refresh time of a complete image, including the scanning of pixels across multiple lines, as well as the time for the top and bottom borders, front header, and back header. During the field scan cycle, the field synchronization signal occupies 2 pixels of time, the trailing edge occupies 25 pixels of time, the top and bottom borders each occupy 8 pixels of time, the effective image occupies 480 pixels of time, and the front shoulder occupies 2 pixels of time.

[0139] (12) Display driver design

[0140] Furthermore, step (12) specifically includes:

[0141] In the system design, the display driver module not only needs to display the acquired image data in real time, but also needs to design a corresponding screen display framework to display the apple detection image, the corresponding grade apple image, and the specific grade in real time. Considering the size of the apple, the system is designed with a 224x224 area size to display the detection image area and the image of the same grade. Specifically, the row and column ranges of the image of the fruit to be evaluated are both set to [100, 323], representing from row 100 to row 323 and from column 100 to column 323. The row range of the image processed by the fruit grading model is [0, 223], representing from row 0 to row 223; the column range is [400, 623], representing from column 400 to column 623. The original size of the image of the target grade of the fruit to be evaluated is 100x100, so an area of ​​the same size is selected for display on the screen. The specific row range is [0, 99], representing row 0 to row 99; the column range is [100, 199], representing column 100 to column 199. Since the image of the target grade of the fruit to be evaluated and the image processed by the fruit grading model are not generated in real time, four memory modules with a width of 1 and a depth of 10000 are needed to store the digital images of grades 1 to 4 in advance, and four memory modules with a width of 1 and a depth of 50176 are needed to store images of the same grade. The display driver module can be called according to the required grading level.

[0142] (2) Overall system setup

[0143] Based on this, a program can be programmed according to the design of the display section to realize the display on the display. Specifically, according to the range of display on the display in (12), the image of the fruit to be evaluated, the image processed by the fruit grade evaluation model, and the target grade of the fruit to be evaluated can be output and displayed.

[0144] Among them, the present invention provides a fruit grading system based on FPGA, combined with Figure 5 From a functional perspective, it can be divided into an image acquisition module, an image processing module, an image grading module, and an image display module. The specific implementation details are as follows:

[0145] Image acquisition module: The system uses an image sensor to acquire images of Fuji apples of different grades and angles in real time to obtain images of the fruit to be evaluated, thus providing high-quality input data.

[0146] Image processing module: This module is used to preprocess the fruit images to be evaluated mentioned above, and to determine the second grade of the fruit images to be evaluated. This module uses image processing algorithms to eliminate noise interference and enhance feature information in order to improve image quality, highlight image features, and at the same time achieve accurate grading of color standards.

[0147] Image grading module: This module is used to process the pre-trained fruit grading model mentioned above to obtain the first grade of the fruit to be graded. This module uses a grading algorithm that combines image processing and deep learning to perform feature extraction and image grading, so as to achieve accurate and efficient processing of the automatic grading task of apples.

[0148] Image display module: This module is used to output and display the images of the fruit to be evaluated, the images processed by the fruit grading model, and the target grade of the fruit to be evaluated. This module can display the grading results to the user in real time via LCD. The interface is required to be intuitive and clear and to effectively display the grading information.

[0149] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A fruit grading method based on FPGA, characterized in that, include: Step 1: Acquire images of the fruit to be evaluated; Step 2: Based on FPGA, the image of the fruit to be evaluated is recognized to determine the first grade of the fruit; Step 2.1: Preprocess the image of the fruit to be evaluated; Step 2.1.1: Calculate the grayscale values ​​of the pixels in the image of the fruit to be evaluated to obtain the grayscale image; Step 2.1.2: Process the grayscale image using a median filtering noise reduction algorithm to obtain the first image; Step 2.1.3: The first image is processed using an 8-direction convolution factor and an improved Sobel sharpening algorithm to obtain the second image; Step 2.1.4: Binarize the second image to obtain the preprocessed image; Step 2.2: The preprocessed image is processed using a pre-trained fruit grading model to obtain the first grade of the fruit to be graded. The first grade represents the contour grade of the fruit to be graded. The fruit grading model is trained based on training samples, which include input samples and output samples. The input samples are fruit images, and the output samples are the contour grades of the fruit images. The fruit grading model is an improvement on the Lenet-5 network and includes convolutional layers, activation layers, pooling layers, and fully connected layers. Step 3: Based on the RGB values ​​of each pixel in the image of the fruit to be evaluated, calculate the hue, lightness, and saturation values ​​of each pixel to determine the target color of the fruit to be evaluated. Based on the preset threshold ranges for hue, lightness, and saturation values ​​of the target color, as well as the hue, lightness, and saturation values ​​of each pixel in the image of the fruit to be evaluated, determine whether each pixel in the image of the fruit to be evaluated belongs to the target color. Based on the ratio of the number of pixels in the fruit to be evaluated that belong to the target color to the total number of pixels in the image of the fruit to be evaluated, determine the second grade of the fruit to be evaluated. Step 4: In the preset grading table, determine the target grade of the fruit to be graded based on the first grade and the second grade; Step 5: Output the image of the fruit to be evaluated, the image processed by the fruit grading model, and the target grade of the fruit to be evaluated; In particular, the calculations in steps 2 to 4 are all performed in the FPGA using parallel and pipelined computing methods.

2. The FPGA-based fruit grading method according to claim 1, characterized in that, The fruit grading model comprises convolutional layers C1, C2, C3, and C4. C1 contains 30 convolutional kernels and has 30 channels. C2 contains 30 convolutional kernels and has 30 channels. C3 contains 16 convolutional kernels and has 16 channels. C4 contains 16 convolutional kernels and has 16 channels. The activation layer uses the LeakyReLU function. The pooling layers include S1 and S2. The fully connected layers consist of a first fully connected layer, a second fully connected layer, and a third fully connected layer. The first fully connected layer contains 120 neurons, the second fully connected layer contains 84 neurons, and the third fully connected layer contains 4 neurons. Based on this, step 2.2 specifically includes: The preprocessed image is input sequentially into C1, activation layer, C2, activation layer, S1, C3, activation layer, C4, activation layer, S2, first fully connected layer, second fully connected layer, and third fully connected layer to obtain the first grade of the fruit to be evaluated.

3. The FPGA-based fruit grading method according to claim 1, characterized in that, Step 2.1 specifically includes: Step 2.1.1: Calculate the grayscale values ​​of the pixels in the image of the fruit to be evaluated to obtain the grayscale image; Step 2.1.2: Definition 3 A sliding window of type 3 is used to slide in the grayscale image. For the grayscale values ​​of the pixels in the 3 rows and 3 columns of the sliding window, the maximum value, the median value, and the minimum value of the grayscale values ​​in each row are determined. The minimum value among the three maximum values, the median value among the three median values, and the maximum value among the three minimum values ​​are obtained. Then, the median value between the minimum value among the three maximum values, the median value among the three median values, and the maximum value among the three minimum values ​​is obtained as the first median value. It is determined whether the pixels in the 2 rows and 2 columns of the sliding window are in the outline position of the fruit to be evaluated. If the pixels in the 2 rows and 2 columns of the sliding window are in the outline position of the fruit to be evaluated, the grayscale value of the pixels in the 2 rows and 2 columns of the sliding window remains unchanged. If the pixels in the 2 rows and 2 columns of the sliding window are not in the outline position of the fruit to be evaluated, the grayscale value of the pixels in the 2 rows and 2 columns of the sliding window is replaced with the first median value. The sliding window is moved so that each pixel in the grayscale image undergoes the above operations. After all pixels have undergone the above operations, the first image is obtained. Step 2.1.3: For 3 The grayscale value of the first image in the sliding window is calculated based on eight preset sets of different weight parameters to obtain a second intermediate value. Each set of weight parameters has a value of 3. The matrix of 3 is used to replace the grayscale values ​​of the pixels in the 2nd row and 2nd column of the sliding window with the second intermediate value. The sliding window is moved so that the above operation is performed on every pixel in the first image. After all pixels have undergone the above operation, the second image is obtained. Step 2.1.4: Binarize the second image to obtain the preprocessed image.

4. The FPGA-based fruit grading method according to claim 2, characterized in that, Step 2.1.1 specifically includes: For each pixel in the image of the fruit to be evaluated, the grayscale value is calculated using the following formula: in, The coordinates of the pixels in the image of the fruit to be evaluated. The red component of the pixels in the image of the fruit to be evaluated. The green component of the pixels in the image of the fruit to be evaluated. The blue component of the pixels in the image of the fruit to be evaluated. The grayscale value of the pixel in the image of the fruit to be evaluated.

5. The FPGA-based fruit grading method according to claim 1, characterized in that, Step 3 involves calculating the hue, brightness, and saturation values ​​of each pixel in the image of the fruit to be evaluated, based on their RGB values, to determine the target color of the fruit. This process specifically includes: For each pixel, obtain the maximum and minimum values ​​of the R, G, and B values. Based on these values, calculate the hue, lightness, and saturation values ​​for each pixel using the following formulas: Where H is the hue value of the pixel, S is the saturation value of the pixel, V is the lightness value of the pixel, G is the G value, B is the B value, R is the R value, min is the minimum value among R, G, and B values, and max is the maximum value among R, G, and B values.

6. The FPGA-based fruit grading method according to claim 1, characterized in that, The fruit grading model is deployed on FPGA hardware. The weight parameters are converted from floating-point data to fixed-point data using the following formula: in, For fixed-point data of weight parameters, To round up, Z represents the scaling factor, where Z is an integer. The weight parameters are floating-point data.

7. A fruit grading system based on FPGA, used to implement the fruit grading method based on FPGA as described in claim 1, characterized in that, Includes image acquisition sensors, FPGAs, and displays; The image acquisition sensor is used to acquire images of the fruit to be evaluated and transmit the acquired images of the fruit to be evaluated to the FPGA; The FPGA is used to receive images of the fruit to be evaluated, implement the contents of steps 2 to 4 in claim 1, and transmit the image processed by the fruit grading model and the target grade of the fruit to be evaluated to the display. The display is used to output and display the image of the fruit to be evaluated, the image processed by the fruit grading model, and the target grade of the fruit to be evaluated.

8. The FPGA-based fruit grading system according to claim 7, characterized in that, When the display outputs the image of the fruit to be evaluated, the image processed by the fruit grading model, and the target grade of the fruit to be evaluated, the row and column ranges of the image of the fruit to be evaluated on the display screen are both set to [100, 323], representing from row 100 to row 323 and from column 100 to column 323. The row range of the image processed by the fruit grading model is [0, 223], representing from row 0 to row 223; the column range is [400, 623], representing from column 400 to column 623. The row range of the target grade of the fruit to be evaluated is [0, 99], representing from row 0 to row 99; the column range is [100, 199], representing from column 100 to column 199.

Images