Blueberry intelligent screening device based on magnetic suction sensing and visual identification technology

Abstract

The invention discloses an intelligent blueberry screening device based on a magnetic suction sensing and visual identification technology, and belongs to the field of intelligent agriculture and automatic sorting systems. The method specifically comprises the steps that a batch of blueberries are conveyed to a screening device, magnets in all meshes are located below the meshes to block mesh channels according to the principle that the magnets attract one another through magnetism according to a sensor instruction, the blueberries with the diameters smaller than 12 mm, larger than 12 mm, smaller than 16 mm and larger than 16 mm are screened out step by step, and finally the blueberries with the diameters larger than 16 mm are left on a screening plate; a camera of the real-time visual monitoring system photographs the blueberries left on the sieve plate, and the maturity state of the blueberries and whether rotting signs exist or not are accurately recognized through a deep learning algorithm and continuous training; carrying out target tracking by utilizing a Deepsort algorithm; raspberry Pi is adopted as a calculation and transmission platform to be in butt joint with a self-resetting aperture screening system and a magnetic relay switch system, and a magnet switch of rotten and immature blueberries is turned on to enable the blueberries to fall off from a sieve plate. Results show that by integrating multiple advanced technologies, high-precision recognition, continuous tracking and automatic sorting of blueberries are achieved, the sorting efficiency and precision are improved, the loss rate is reduced, the product quality is improved, and the adaptability and flexibility of the system are enhanced.

Description

Technical Field

[0001] The technical field of this research and development achievement mainly focuses on smart agriculture and automated sorting systems.

[0002] Specifically, the technologies involved include: intelligent sorting technology, visual recognition technology, electromagnetic technology, embedded system technology, data processing and communication technology, single-chip control technology, mechanical structure innovation and algorithm research. Background Art

[0003] Blueberry, as a fruit with extremely high nutritional value, is experiencing a continuous increase in global market demand, a trend that has greatly promoted the rapid development of the blueberry industry. However, as the scale of the industry continues to expand, problems in the post-harvest sorting process have gradually become prominent, becoming a key bottleneck restricting the further upgrading of the blueberry industry. Efficient and accurate sorting can not only significantly increase the commodity value of blueberries, but also ensure the stability of the supply chain, thereby better meeting consumers' high standards for food safety and quality. However, the current traditional manual sorting method has been unable to adapt to the development needs of the modern blueberry industry due to problems such as low efficiency, high error rate and high cost.

[0004] The lag in sorting technology directly hinders the homogenization and standardization of blueberry product quality. At present, domestic blueberry sorting mainly relies on manual operation, which not only has low detection efficiency, but is also easily affected by subjective judgment, making it difficult to form a unified high-quality standard. This situation not only weakens the market competitiveness of blueberry products and limits their market expansion at home and abroad, but also makes it difficult to establish a stable market reputation and brand image due to quality fluctuations. In the long run, this will seriously restrict the internationalization process of the blueberry industry.

[0005] In addition, the lack of mechanization and automation is another major challenge facing blueberry sorting technology. Compared with advanced foreign sorting equipment and technology, domestic equipment is generally backward and has a low degree of automation, which not only greatly increases production costs and reduces production efficiency, but also has a significant impact on preservation and logistics. Specifically, the lack of mechanization and automation leads to a long sorting process, which increases the time it takes for blueberries to be picked and circulated in the market, thus affecting the freshness and taste of the product. This problem is particularly prominent when the cold chain logistics system is still imperfect, further shortening the shelf life of blueberries and reducing their market value. At the same time, the lagging level of mechanization and automation also limits the large-scale development of the blueberry industry, making it difficult to meet the growing market demand for large-scale production and rapid circulation.

[0006] In summary, the backwardness of sorting technology, the lack of a standardized system, and the insufficient level of mechanization and automation have become deep-seated obstacles to the high-quality development of the blueberry industry. Therefore, strengthening the research and development and innovation of sorting technology and improving the level of mechanization and automation have become the key paths to enhance the competitiveness of the blueberry industry, meet market demand, and promote the sustainable and healthy development of the industry.

[0007] At present, the blueberry sorting systems on the market are mainly based on mechanical design, automation technology and computer vision technology. Among them, mechanical design technology uses vibration principles and screen-type belts to achieve size grading; automation technology improves sorting efficiency and reduces manual intervention; computer vision technology uses intelligent vision systems to quickly and accurately classify blueberries. However, although these technologies have significantly improved sorting efficiency and accuracy, there are still many shortcomings, such as fruit damage caused by transmission methods, inefficient loading and sorting devices, and great damage to fruit quality by grading and unloading devices. In addition, the shortcomings and poor adaptability of color sorting technology are also shortcomings of existing technologies. Summary of the invention

[0008] This invention is aimed at blueberry sorting, and designs a blueberry intelligent screening device based on magnetic suction sensing and visual recognition technology. In the automated blueberry sorting system, the magnetic suction electrical switch device controls the high-level output of the row and column lines through the dual single-chip microcomputer, activates the electromagnet at a specific coordinate, and opens the mesh switch for screening. After sorting, the dual single-chip microcomputer outputs a low level to demagnetize the electromagnet, and the mesh is restored to a diameter of 12mm under the action of the spring. Then the real-time target detection and tracking of the blueberry image, YOLOv8 outputs the bounding box through non-maximum suppression, and the DeepSORT algorithm calculates the intersection and union ratio of the new and old target boxes, optimizes the matching with the Hungarian algorithm, updates the Kalman filter, and achieves accurate tracking. The data transmission interface between the Raspberry Pi and the single-chip microcomputer is constructed through serial communication and other methods to ensure real-time performance and error handling mechanism. Finally, the system uses dual single-chip microcomputers to control the electromagnet to achieve effective screening of immature or rotten blueberries.

[0009] The present invention focuses on the combination of sensors and mechanical devices and the application of visual recognition in the context of blueberry sorting. The core is to use the output and input of gate circuits as a bridge to fully utilize the advantages of automated mechanical devices with the help of sensors, and to accurately analyze and process blueberry images, so as to determine the most suitable visual recognition strategy in the specific application context of blueberry quality grading, opening up an effective path for blueberry sorting and providing a valuable reference idea. The technical solution adopted by the present invention covers the following four key stages:

[0010] (1) When the magnetic attraction relay switch device is working, the dual single-chip microcomputer outputs a high level to the row and column lines. After the AND gate circuit operation, the electromagnet at the specific coordinate is turned on to absorb the adjacent magnet sheet and open the corresponding mesh (such as 22mm aperture to screen defective products). After the three-level sorting, the dual single-chip microcomputer outputs a low level, the electromagnet is demagnetized, and the mesh restores the initial screening diameter of 12mm under the action of the torsion spring.

[0011] (2) The YOLOv8 single-stage target detection algorithm is based on a deep convolutional neural network, which contains 19 convolutional layers and 5 pooling layers to extract features. The fully connected layer maps the category probability and location information, and outputs the bounding box with non-maximum suppression. It can detect in real time and is good at detecting small objects. First, collect a dataset of labeled blueberry images, divide them into training sets and validation sets, and preprocess and enhance them. Select the YOLOv8s model for training, iteratively update the weights, and evaluate them based on the validation set. After the training is completed, the camera collects images, the microprocessor uses the model for analysis, and the Raspberry Pi is used as a computing and transmission platform to achieve blueberry recognition and tracking.

[0012] The calculation formula is as follows (2).

[0013]

[0014] (3) When a new video frame appears, DeepSORT calculates the association cost between the newly detected target and the existing tracking trajectory, and then determines the best association scheme through the optimization algorithm. If the match is successful, the tracking trajectory status information is updated; if the target is not matched for a long time, the tracking is terminated. DeepSORT constructs a similarity matrix through the intersection and union ratio of the target boxes of the previous and next two frames, and then uses the Hungarian algorithm for matching and updating the Kalman filter.

[0015] (4) Build a reliable data transmission interface between the Raspberry Pi and the MCU. You can choose serial communication (such as UART), I2C, SPI or wireless communication (Wi-Fi, Bluetooth) and determine the data format. Write a program on the MCU to receive the Raspberry Pi data and perform operations to ensure that the data transmission and processing time meet the real-time requirements. At the same time, add error detection and processing mechanisms during data transmission and processing to ensure system stability. In addition, use two MCUs to output high levels to the row and column lines to control the conduction of the electromagnet, suck away the magnet sheet, and accurately control the mesh to achieve the screening and transportation of unripe or rotten blueberries (less than 22 mm in diameter).

[0016] Beneficial Effects

[0017] (1) The present invention combines YOLOv8 with DeepSORT algorithm to achieve high-precision recognition and continuous tracking of blueberries, significantly improving the accuracy and efficiency of sorting.

[0018] (2) The present invention uses a real-time visual monitoring system and a deep learning algorithm to accurately extract information such as the maturity, decay state, and skin integrity of blueberries from high-definition images, ensuring high accuracy.

[0019] (3) The present invention has high sorting accuracy and low operating cost on the basis of achieving fully automated operation.

[0020] (4) The present invention has significantly improved the sorting accuracy compared with the traditional sorting method; in terms of adaptability and flexibility, compared with other sorting systems, it can more widely meet the sorting needs of different products, and through the support of advanced technologies such as Raspberry Pi and modular design, a more efficient and accurate sorting process is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 The overall scheme framework diagram of the present invention is

[0022] Figure 2 Device modeling diagram

[0023] Figure 3 Convolution operation process diagram

[0024] Figure 4 Maximum pooling layer operation process diagram

[0025] Figure 5 Yolov8 network model structure diagram

[0026] Figure 6 MCU control schematic diagram

[0027] Figure 7 Model training PR curve and P curve training results

[0028] Figure 8 Upper computer model training prediction result diagram

[0029] Fig. 9 Deepsort algorithm flow chart

[0030] Fig.10 Flowchart of Raspberry Pi transmission platform construction

[0031] Fig.11 Vision function flow chart

[0032] Fig.12 Blueberry defect map

[0033] Fig.13 Image set training flow chart

[0034] Fig.14 Image testing flow chart

[0035] Fig.15 Raspberry Pi module diagram

[0036] Best Mode for Carrying Out the Invention

[0037] The present invention is further described in detail below with reference to the accompanying drawings and examples.

[0038] The present invention provides a blueberry intelligent screening device based on magnetic sensing and visual recognition technology, comprising:

[0039] S1. A batch of blueberries is transported to the screener. The magnets in each mesh block the mesh channel under the mesh according to the sensor instructions through the principle of magnetic attraction. Blueberries with diameters less than 12mm, greater than 12mm and less than 16mm, and greater than 16mm are screened step by step. Finally, blueberries with diameters greater than 16mm are left on the screen plate.

[0040] S2. The camera of the real-time visual monitoring system takes pictures of the blueberries left on the sieve plate, and accurately identifies the maturity of the blueberries and whether there are signs of decay through deep learning algorithms and continuous training;

[0041] S3. Use Deepsort algorithm to track the target;

[0042] S4. Use Raspberry Pi as a computing and transmission platform to connect to the self-resetting aperture screening system and magnetic suction relay switch system, and activate the magnet switch of abnormal blueberries to remove them.

[0043] In step S1 of this embodiment, a batch of blueberries is transported to the screener. The magnets in each mesh block the mesh channel under the mesh according to the sensor instruction and the principle of magnetic attraction. Blueberries with diameters less than 12 mm, greater than 12 mm and less than 16 mm, and greater than 16 mm are screened out step by step. Finally, blueberries with a diameter greater than 16 mm are left on the screen plate, including:

[0044] The experimental device designed by the present invention has an off-axis external cylindrical electromagnetic switch, a relay, and a circular sieve plate with several circular holes with a diameter of 22 mm. Figure 2 shown.

[0045] Select two magnets with diameters of 6mm and 8mm respectively, and drill a small hole with a diameter of 1.5mm near the edge of the magnet as its rotation axis. Drill holes at both ends of the same circumference of the screen plate, insert positioning bolts (in order to prevent the magnetic fields between adjacent magnets from interfering with each other, the adjacent magnets must be arranged in a diameter direction perpendicular to each other), and paste electromagnets on the left and right opposite to the magnets.

[0046] The magnet is fixed on the sieve plate through the shaft. When the electromagnet is energized by the sensor command and attracts the magnet, the left and right iron sheets block the round holes respectively, so that the maximum distance of the magnet blocking the sieve hole is 4mm and 6mm respectively. In this way, the effective diameter of the sieve hole of the screen can be adjusted to three different gears of 12mm, 16mm and 22mm, thus realizing the effective grading and screening of blueberries of different sizes.

[0047] By outputting high-level signals to the row and column lines through the two microcontrollers, the conduction state of each pair of electromagnets can be precisely controlled to absorb the corresponding magnet pieces. When the two microcontrollers work in coordination to precisely control the mesh holes at the specified coordinates, blueberries of a specific size can be separated and transported to the next level of processing (the diameter of the mesh holes can reach 22 mm at this time, ensuring that all unripe or rotten blueberries can be screened out).

[0048] The specific implementation functions are as follows:

[0049] The two microcontrollers each output a high-level signal to the corresponding pin, thereby activating a row line and a column line at the same time. The electromagnet at each mesh performs a logical operation on the received row signal and column signal through an AND gate logic circuit. Only when both are high level will it receive a high-level signal, and then accurately control the electromagnet at that position to conduct, causing the adjacent magnet piece to be sucked away from the original position, so that the two switches of the coordinate mesh are opened at the same time.

[0050] The two single-chip microcomputers control each row and column to output a low level respectively. The output result of each electromagnet and gate circuit operation is a low level, thereby demagnetizing the electromagnet, which loses its attraction to the magnet sheet, demagnetizing the mesh iron sheet and restoring it to a state of mutual attraction, thereby closing the switch, that is, resetting the device.

[0051] After completing the three-level sorting work, the two single-chip microcomputers control each row and column to output a low level respectively. The output result of each electromagnet and gate circuit operation is a low level, thereby demagnetizing the electromagnet, which loses its attraction to the magnet sheet. At this time, the magnet sheet is only affected by the torsion spring and the magnetic force between each pair of magnet sheets, and returns to the position where the two magnet sheets are closest to each other (so that the initial screenable diameter of the mesh is restored to 12mm).

[0052] In step S2 of this embodiment, the camera of the real-time visual monitoring system takes pictures of the blueberries on the sieve plate, and accurately identifies the maturity state of the blueberries and whether there are signs of decay through deep learning algorithms and continuous training, including:

[0053] Deep learning is a machine learning technology that relies on the construction and training of multi-layer neural networks to achieve data learning and pattern recognition tasks. Through a multi-layer mapping mechanism, deep learning can automatically capture complex data features and reveal high-level information in the data.

[0054] Among the many models of deep learning, convolutional neural network (CNN) occupies a pivotal position. CNN is usually composed of multiple convolutional layers and a fully connected layer at the top, and also incorporates structures such as weight sharing and pooling layers. At present, CNN has shown a wide range of application value in many fields such as image recognition, speech recognition and target detection.

[0055] In the CNN architecture, the convolution layer plays a core role in extracting key features from the input data. By performing a weighted sum operation on the local area of ​​the input data and using the sliding window technology, the convolution layer can efficiently capture the local features of the data. As the network level goes deeper, these local features will gradually be transformed into more abstract and advanced feature representations.

[0056] The convolution operation process is as follows Figure 3 shown.

[0057] The two-dimensional convolution layer is an important component for processing image data, and its calculation method can be expressed by formula (1).

[0058]

[0059] In the convolutional neural network architecture, the convolution kernel plays the role of performing convolution operations to extract feature information from the input feature map. Specifically, the convolution kernel focuses on a local area of ​​the input feature map (such as Figure 1 The gray part in the image is used for analysis, and the weighted sum of the features in the area and the weight of the convolution kernel is used for analysis. When performing image convolution, edge padding technology is often used to keep the size of the feature map stable. In addition, in order to more effectively extract diverse features, a typical convolution layer will design multiple convolution kernels, each of which is independently responsible for capturing different features of the input feature map. As the network training progresses, the weights of these convolution kernels are automatically optimized to better meet the needs of specific tasks. Figure 1 The output of the convolution is shown in formula (2).

[0060] output=1*(-1)+2*0+…+5*1+2*(-1)+3*0+6*1=4 (2)

[0061] The introduction of Sigmoid nonlinear activation function makes it easier for neural networks to adapt to different types of data. It helps neural networks to better perform nonlinear modeling and feature extraction. The Sigmoid formula is shown in formula (3).

[0062]

[0063] The pooling layer is a downsampling layer in a neural network. It aims to reduce network parameters by dividing the feature map into regions and reducing the dimension. This layer does not change the overall scale of the model. Although the dimension reduction in the pooling process may lead to the loss of some feature information, it is a necessary compromise. As a widely used pooling method, max pooling is particularly suitable for scenarios that need to highlight significant features. For example, in image classification tasks, the max pooling layer can effectively extract core features such as texture and edges of the image.

[0064] like Figure 4 As shown in the figure, the pooling layer selects the maximum value for each sub-region as the output. The new matrix formed by these maximum values ​​corresponds to the input data, which has the advantages of reducing the risk of overfitting, efficient feature extraction and reducing the computational burden. The fully connected layer is widely used in deep learning. Its nodes are connected to all nodes in the previous layer, integrating feature information to form a new high-dimensional feature representation. Usually, the fully connected layer is located after the convolutional layer or other feature extraction layer, and its output is used as the final classification or regression prediction of the model.

[0065] YOLOv8 is a YOLO model released by Ultralytics in January 2023. It is suitable for various tasks such as target detection, image classification, instance segmentation, and key point detection. Its network structure is shown in the figure below: Figure 5 shown.

[0066] The model consists of four major components: input processing (Input), feature extraction backbone (Backbone), feature fusion neck (Neck) and target detection head (Head). In the input stage, the present invention uses Mosaic data enhancement technology and adapts to model requirements of different sizes by adjusting model hyperparameters.

[0067] The feature extraction backbone is responsible for extracting key features from the input data. The present invention adopts the Darknet-53 architecture, combines the residual structure of the C3 module and the ELAN concept in YOLOv7, and innovatively designs the C2f (CSPLayer_2Conv) module to implement residual learning. In addition, the present invention also introduces the SPPF (Spatial Pyramid Pooling Fusion) module to effectively fuse multi-scale features.

[0068] In the feature fusion neck, the present invention uses the features extracted by the backbone network for deep fusion. Here, the C3 module is replaced by the C2f module to further improve the feature fusion effect. At the same time, the neck network draws on the advanced ideas of PAN20 and FPN21 to construct an efficient feature pyramid structure.

[0069] Finally, in the target detection head part, the present invention determines the category and precise location of the detected target based on the rich feature information obtained by processing in the first two parts. The detection head design contains three feature maps of different sizes, which are respectively used to detect target objects of different sizes, thereby improving the detection accuracy and generalization ability of the model.

[0070] Usually, the precision rate P, recall rate R, the average precision rate AP of a single class label, the average precision rate mAP of all class labels and the detection rate FPS (Frames Per Second) are used to evaluate the performance of the model. AP is the area enclosed by drawing a graph with P and R as the two axes. When there is only one category of research objects, AP and mAP are equivalent, so this article uses mAP as the comparison indicator. The indicator calculation formula is as follows (4).

[0071]

[0072] First, for model training, a large dataset of images containing rotten, unripe, and normally ripe blueberry samples needs to be collected and labeled. The blueberries in each image need to be annotated, including drawing bounding boxes and marking corresponding categories (such as rotten, unripe, and ripe). This step is crucial for model learning because the model needs to learn to recognize these features from the image.

[0073] Before training, the dataset needs to be divided into a training set and a validation set in a ratio of 8:2. To improve the efficiency and performance of model training, it may be necessary to preprocess the images, such as cropping, scaling, and normalization, and use data enhancement techniques (such as random rotation and flipping) to increase the diversity of the dataset and the generalization ability of the model.

[0074] Secondly, select the YOLOv8 model as the starting point and set 100 training cycles (Epochs). Based on the pre-trained model, input the training set for iterative training. By continuously updating the weight parameters and using the stochastic gradient descent (SGD) optimization algorithm, the loss function value is gradually reduced, thereby improving the model prediction performance. After each training cycle, the model performance is evaluated using the validation set. When the model reaches the preset number of training steps or goals, save the model for subsequent reasoning.

[0075] The camera collects image data within the visible range, and the microprocessor analyzes the image data collected by the camera through the yolo-v8 model to determine whether the blueberries are green, rotten, or spoiled. The data calculated by the model is displayed on the LCD. The model will output a bounding box containing the location and category of the blueberries, thereby realizing the identification of rotten and unripe blueberries.

[0076] Raspberry Pi plays a vital role in the scenario where YOLOv8 and DeepSORT algorithms are used to identify and track rotten and unripe blueberries on the conveyor belt, while using Raspberry Pi as a computing and transmission platform.

[0077] In step S3 of this embodiment, using the Deepsort algorithm to track the target includes:

[0078] The DeepSORT algorithm extracts target visual features through deep learning. Its core advantage lies in using deep learning models to extract visual features that are richer and more discriminative than traditional features. These features can effectively distinguish visually similar targets, such as blueberries of different shapes, sizes or similar colors.

[0079] In the blueberry sorting scenario, deep learning models (such as convolutional neural networks (CNNs)) are trained on key features of blueberry images, including contours, color distribution, and texture details. These features are used to build a "feature descriptor" for each detected blueberry, allowing the tracking algorithm to accurately distinguish and track each individual blueberry target. The specific process is as follows: Fig.11 shown.

[0080] Whenever a new video frame appears, DeepSORT calculates the "association cost" between the new detection and the existing tracking trajectory, which takes into account multiple dimensions such as position, speed difference, and similarity of visual features. Then, an optimization algorithm (such as the Hungarian algorithm) is used to determine the best data association scheme, that is, to find the detection and trajectory matches that are most likely to represent the same target.

[0081] After a successful match, DeepSORT will update the status information of the corresponding tracking track (such as position, velocity, acceleration, and visual feature descriptors) and continue to track the target. If a tracking track does not match any detection for a long time, it may be determined that the target has left the field of view and the tracking will be terminated.

[0082] First, the motion state of the target is predicted by Kalman filtering to obtain the position and state information of the target, which is expressed as (x, y, w, h, vx, vy, vw, vh). Among them: x and y represent the center coordinates of the detection frame; w and h represent the aspect ratio and height of the prediction frame respectively; vx, vy, vw, and vh represent the relative change speed of each parameter x, y, w, and h respectively. Then, the Mahalanobis distance is used to judge the correlation between the target detected by MSB-YOLOv7 and the target predicted by Kalman filtering, and d is used to calculate the correlation between the target detected by MSB-YOLOv7 and the target predicted by Kalman filtering. (1) (i,j) represents the Mahalanobis distance between the i-th prediction box and the j-th detection box, calculated as (5).

[0083]

[0084] Where: d j Represents the state vector of the jth detection box; y i Represents the state vector of the i-th prediction box; Represents the inverse covariance matrix of the detection results and tracking results.

[0085] When the target is blocked for a long time or the viewing angle is displaced, it is necessary to introduce appearance information and use cosine distance to solve the ID switching problem caused by the blockage. (2) (i,j) represents the cosine distance between the i-th prediction box and the j-th detection box, which is calculated as (6):

[0086]

[0087] Where: r j is d j The eigenvector of represents the i-th feature vector saved in the tracker; R i It is the appearance feature vector library.

[0088] In order to achieve the complementary advantages of the short-term position information measured by the Mahalanobis distance and the long-term occluded target information represented by the cosine distance, the above two distances are summed in a weighted manner to obtain the final metric value c i,j , the calculation formula is (7)

[0089] c i,j =λd (1) (i,j)+(1-λ)d (2) (i,j) (7)

[0090] Where λ is a hyperparameter. If and only if c i,j Between (1) (i,j) and d (2) (i,j) is considered to be associated with the target.

[0091] The DeepSort tracking algorithm constructs a similarity matrix for the target with a certain state through the intersection over union (IoU) between the target frames of the previous and next two frames, and then matches them through the Hungarian algorithm to complete the update of the Kalman filter.

[0092] In step S4 of this embodiment, a Raspberry Pi is used as a computing and transmission platform to connect to the self-resetting aperture screening system and the magnetic attraction relay switch system, and the magnet switch of abnormal blueberries is activated to remove them.

[0093] Raspberry Pi is used as a computing platform to run YOLOv8 and DeepSORT algorithms. The platform construction flow chart is as follows: Fig.12 .

[0094] Establish a reliable data transmission interface between the Raspberry Pi and the microcontroller. This can be serial communication (such as UART), I2C, SPI, or wireless communication (such as Wi-Fi or Bluetooth). Determine a data format that the Raspberry Pi can generate data in this format and the microcontroller can parse and utilize the data. Write a program on the microcontroller so that it can receive data from the Raspberry Pi and perform corresponding operations based on the data. Ensure that the time for data transmission and processing is within an acceptable range to meet real-time requirements.

[0095] Add error detection and handling mechanisms during data transmission and processing to ensure system stability.

[0096] The two single-chip microcomputers output high levels to the row and column lines to accurately control the conduction of each pair of electromagnets, so that the pair of magnets are sucked away. The two single-chip microcomputers cooperate to accurately control the mesh holes at the coordinates, so that the blueberries are separated and transported to the next level (at this time, the mesh holes can be 22mm in diameter, which can filter out all unripe or rotten blueberries of all sizes).

Claims

1. A blueberry intelligent screening device based on magnetic sensing and visual recognition technology, characterized in that: include: S1. A batch of blueberries is transported to the screener. The magnets in each mesh block the mesh channel under the mesh according to the sensor instructions through the principle of magnetic attraction. Blueberries with diameters less than 12mm, greater than 12mm and less than 16mm, and greater than 16mm are screened step by step. Finally, blueberries with diameters greater than 16mm are left on the screen plate. S2. The camera of the real-time visual monitoring system takes pictures of the blueberries left on the sieve plate, and accurately identifies the maturity of the blueberries and whether there are signs of decay through deep learning algorithms and continuous training; S3. Use Deepsort algorithm to track the target; S4. Use Raspberry Pi as a computing and transmission platform to connect to the self-resetting aperture screening system and magnetic suction relay switch system, and activate the magnet switch of abnormal blueberries to remove them.

2. The visual recognition method based on the combination of YOLOv8 algorithm, DeepSORT algorithm and Raspberry Pi as claimed in claim 1, characterized in that: A batch of blueberries is transported to the screener. The magnets in each mesh block the mesh channel by the principle of magnetic attraction according to the sensor instructions, and gradually screen out blueberries with diameters less than 12mm, greater than 12mm and less than 16mm, and greater than 16mm. Finally, blueberries with diameters greater than 16mm are left on the screen: When the magnetic attraction relay switch device is working, the dual single-chip microcomputer outputs a high level to the row and column lines. After the AND gate circuit operation, the electromagnet at the specific coordinate is turned on to absorb the adjacent magnet sheet and open the corresponding mesh (such as 22mm aperture to screen defective products). After the three-level sorting, the dual single-chip microcomputer outputs a low level, the electromagnet is demagnetized, and the mesh restores the initial screening diameter of 12mm under the action of the torsion spring.

3. The visual recognition method based on the combination of YOLOv8 algorithm, DeepSORT algorithm and Raspberry Pi as claimed in claim 1, characterized in that: The camera of the real-time visual monitoring system takes pictures of the blueberries on the sieve plate and accurately identifies the maturity of the blueberries and whether there are signs of decay through deep learning algorithms and continuous training: The YOLOv8 single-stage target detection algorithm is based on a deep convolutional neural network, with 19 convolutional layers and 5 pooling layers to extract features, and maps the category probability and location information through a fully connected layer, outputting bounding boxes with non-maximum suppression, which can detect in real time and is good at detecting small objects. First, collect a dataset of labeled blueberry images, divide them into training sets and validation sets, and preprocess and enhance them. Select the YOLOv8s model for training, iteratively update the weights, and evaluate them based on the validation set. After the training is completed, the camera collects images, the microprocessor uses the model for analysis, and the Raspberry Pi is used as a computing and transmission platform to achieve blueberry recognition and tracking. The calculation formula is as follows (1).

4. The visual recognition method based on the combination of YOLOv8 algorithm, DeepSORT algorithm and Raspberry Pi as claimed in claim 1, characterized in that: Use DeepSORT algorithm for target tracking; When a new video frame appears, DeepSORT calculates the association cost between the newly detected target and the existing tracking trajectory, and then determines the best association solution through the optimization algorithm. If the match is successful, the tracking trajectory status information is updated; if the target is not matched for a long time, the tracking is terminated. DeepSORT constructs a similarity matrix through the intersection and union ratio of the target frames of the previous and next two frames, and then uses the Hungarian algorithm to match and update the Kalman filter.

5. The visual recognition method based on the combination of YOLOv8 algorithm, DeepSORT algorithm and Raspberry Pi as claimed in claim 1, characterized in that: Raspberry Pi is used as a computing and transmission platform to connect to the self-resetting aperture screening system and magnetic suction relay switch system, and the magnet switch of abnormal blueberries is activated to remove them: Build a reliable data transmission interface between the Raspberry Pi and the MCU. You can choose serial communication (such as UART), I2C, SPI or wireless communication (Wi-Fi, Bluetooth) and determine the data format. Write a program on the MCU to receive the Raspberry Pi data and perform operations to ensure that the data transmission and processing time meet the real-time requirements. At the same time, add error detection and processing mechanisms during data transmission and processing to ensure system stability. In addition, use two MCUs to output high levels to the row and column lines to control the conduction of the electromagnet, suck away the magnet sheet, and accurately control the mesh to achieve the screening and transportation of unripe or rotten blueberries (less than 22mm in diameter).

Images