Fruit type and rot detecting and sorting system based on deep learning

Abstract

The invention relates to the field of deep learning and target detection, in particular to a fruit type and rot detecting and sorting system based on deep learning. A fruit type and rot detecting and sorting system based on deep learning is divided into a target recognition system and a bottom layer control system. The target identification system is carried on a Ubuntu operating system to operate, and the bottom layer control system is carried on an ARM chip as a master control of bottom layer control. The fruit sorting device is mainly used for identifying the types of the fruits and whether the fruits are rotten, and sorting according to the identification result. According to the invention, convenient management can be provided and the labor cost can be saved in the fruit retail sorting process.

Description

Technical Field

[0001] The present invention relates to the field of deep learning and target detection, and in particular to a fruit type and corruption detection and sorting system based on deep learning. Background Art

[0002] In the modern food processing and logistics industry, efficient and accurate identification and sorting of fruits is the key to ensuring food safety and reducing waste. Traditional fruit sorting systems rely on manual inspection or simple mechanical screening, which are not only inefficient but also difficult to ensure consistency and accuracy. With the development of deep learning technology, a fruit type and spoilage detection and sorting system based on deep learning has emerged. The system uses advanced machine learning algorithms of convolutional neural networks to analyze fruit image data to quickly identify fruit types and accurately judge the spoilage state. This system can automatically learn the appearance characteristics of fruits, including color, shape, and texture, so as to achieve efficient and accurate fruit sorting without relying on human intervention. This not only improves sorting efficiency and reduces labor costs, but also helps to improve food safety and reduce economic losses caused by spoiled fruits. Summary of the invention

[0003] The invention aims to efficiently and accurately identify and sort fruits, which is the key to ensuring food safety and reducing waste, and reduce manual inspection or simple mechanical screening. It aims to improve sorting efficiency, reduce labor costs, and also help improve food safety levels and reduce economic losses caused by spoiled fruits.

[0004] In order to achieve the above-mentioned object, the present invention provides a fruit type and spoilage detection and sorting system based on deep learning, which is characterized by comprising a target recognition system and an underlying control system:

[0005] The target recognition system includes a camera module, a preprocessing module, a deep learning module and a first communication module. The camera module, the preprocessing module, the deep learning module and the first communication module are connected in sequence.

[0006] The camera module is used to collect fruit image data in real time;

[0007] The pre-processing module is used to adjust the resolution, brightness and saturation of the fruit image collected in real time;

[0008] The deep learning module is used to perform model training and recognition on the fruit images collected in real time to obtain the type of fruit and whether it is rotten;

[0009] The first communication module is used to transmit the recognition result to the underlying control system.

[0010] The deep learning module includes a video framing module, a Resnet module, a fully connected layer module and a classification data module. The video framing module, the Resnet module, the fully connected layer module and the classification data module are connected in sequence.

[0011] The video framing module is used to divide the real-time collected video stream data into multiple frames of picture data;

[0012] The Resnet module is used as the core of the convolutional neural network recognition of the system;

[0013] The fully connected layer module is used to calculate the probability of each fruit type and whether the fruit is rotten in the image data;

[0014] The classification data module is used to find the value with the highest probability to determine the corresponding fruit type and whether the fruit is rotten.

[0015] Wherein, the bottom control system includes a second communication module, a data receiving module, a main control module, an indicator light module, a motor drive module and a power module. The second communication module, the data receiving module, the main control module, the indicator light module and the motor drive module are connected to the power module respectively. The second communication module, the data receiving module and the main control module are connected in sequence. The motor drive module and the indicator light module are connected to the main control module in sequence respectively.

[0016] The second communication module is used to transmit information data of the first communication module in the target identification system;

[0017] The data receiving module is used to decode and process the character string data in the target recognition system and convert the character string data into a corresponding state to the main control module.

[0018] The main control module is used to process all data in the underlying control system;

[0019] The indicator light module is used to light up the indicator lights corresponding to the fruit type and whether the fruit is rotten, so as to intuitively display the recognition result to the user;

[0020] The motor drive module is used to control the conveyor belt motor for sorting, and deliver the fruit to the location corresponding to the type of fruit and whether it is rotten;

[0021] The power supply module stops supplying power to the second communication module, the data receiving module, the main control module, the indicator light module and the motor driving module. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the implementation of the present invention or the solutions in the prior art, the drawings required for use in the implementation examples or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0023] Figure 1 It is a structural schematic diagram of a fruit type and spoilage detection and sorting system based on deep learning provided by the present invention.

[0024] Figure 2 It is a structural diagram of the target recognition system.

[0025] Figure 3 It is a structural diagram of the deep learning module.

[0026] Figure 4 It is a structural diagram of the underlying control system.

[0027] 1-target recognition system, 2-bottom-level control system, 11-camera module, 12-preprocessing module, 13-deep learning module, 14-first communication module, 131-video framing module, 132-Resnet module, 133-full connection module, 134-classification data module, 2-second communication module, 22-data receiving module, 23-main control module, 24-indicator light module, 25-motor drive module, 26-power supply module. DETAILED DESCRIPTION

[0028] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0029] See also Figures 1 to 4 The present invention provides a fruit type and spoilage detection and sorting system based on deep learning, including a target recognition system 1 and an underlying control system 2:

[0030] The target recognition system 1 comprises a camera module 11, a preprocessing module 12, a deep learning module 13 and a first communication module 14. The camera module 11, the preprocessing module 12, the deep learning module 13 and the first communication module 14 are connected in sequence.

[0031] The camera module 11 is used to collect fruit image data in real time;

[0032] The pre-processing module 12 is used to adjust the resolution, brightness and saturation of the fruit image collected in real time;

[0033] The deep learning module 13 is used to perform model training and recognition on the fruit images collected in real time to obtain the type of fruit and whether it is rotten;

[0034] The first communication module 14 is used to transmit the recognition result to the underlying control system 2.

[0035] In this embodiment, the camera module 11 is a 1080P camera that can collect real-time data and transmit it to the computer equipped with the Ubuntu operating system. The pre-processing data module 12 redefines the real-time image data into rules that can be input to the model. The deep learning module 13 uses a convolutional neural network to perform model training and recognition on the real-time collected fruit images to obtain the type of fruit and whether it is rotten. The first communication module 14 connects the hardware USB through the Python serial port library to transmit the recognition result to the underlying control system.

[0036] Furthermore, the deep learning module 13 includes a video framing module 131, a Resnet module 132, a fully connected layer module 133 and a classification data module 134. The video framing module 131, the Resnet module 132, the fully connected layer module 133 and the classification data module 134 are connected in sequence.

[0037] The video framing module 131 is used to divide the real-time collected video stream data into multiple frames of picture data;

[0038] The Resnet module 132 is used as the core of the convolutional neural network recognition of the system;

[0039] The fully connected layer module 133 is used to calculate the probability of each fruit type and whether the fruit is rotten in the image data;

[0040] The classification data module 134 is used to find the value with the maximum probability to determine the corresponding fruit type and whether the fruit is rotten.

[0041] In this embodiment, the video framing module 131 divides the real-time collected video stream data into multiple frames of image data through the OpenCV library. The Resnet module 132 imports the Resnet105 of the convolutional neural network model as a training model through transfer learning. The fully connected layer module 133 is used to calculate the probability of the image data in each fruit type and whether the fruit is rotten. The classification data module 134 calculates the value with the maximum probability to determine the corresponding fruit type and whether the fruit is rotten, and summarizes and classifies them into strings to display on the terminal of the Ubuntu operating system.

[0042] Further, the bottom control system 2 includes a second communication module 21, a data receiving module 22, a main control module 23, an indicator light module 24, a motor drive module 25 and a power module 26. The second communication module 21, the data receiving module 22, the main control module 23, the indicator light module 24 and the motor drive module 25 are respectively connected to the power module 26. The second communication module 21, the data receiving module 22 and the main control module 23 are connected in sequence. The motor drive module 25 and the indicator light module 24 are respectively connected to the main control module 23 in sequence.

[0043] The second communication module 21 is used to transmit information data of the first communication module 14 in the target recognition system 1;

[0044] The data receiving module 22 is used to decode and process the string data in the target recognition system 1 and convert the string data into a corresponding state to the main control module 23 .

[0045] The main control module 23 is used to process all data in the underlying control system 2;

[0046] The indicator light module 24 is used to light up the indicator lights corresponding to the fruit type and whether the fruit is rotten, so as to intuitively display the recognition result to the user;

[0047] The motor drive module 25 is used to control the conveyor belt motor for sorting, and deliver the fruit to the position corresponding to the type of fruit and whether it is rotten;

[0048] The power supply module 26 stops supplying power to the second communication module 21 , the data receiving module 22 , the main control module 23 , the indicator light module 24 and the motor driving module 25 .

[0049] In this embodiment, the second communication module 21 is connected to the serial port interface of the ARM chip. The data receiving module 22 decodes and processes the data in the second communication module 21. The main control module 23 uses the H7 chip of the STM32 as the main control of the control system. The indicator light module 24 is used to light up the indicator lights corresponding to the fruit type and whether the fruit is rotten, and intuitively display the recognition result to the user. The motor drive module 25 controls the conveyor belt motor of the sorting by connecting the RM2006 and RM3508 motors through CAN communication.

[0050] Real-time example:

[0051] First, the target recognition system is installed on the Ubuntu operating system. The camera is opened through the OpenCV library control. The real-time fruit image data on the conveyor belt is preprocessed and imported into the Resnet105 convolutional neural network model. By collecting sufficient data sets, the model can classify the type of fruit and whether it is rotten. The recognition results are printed on the Ubuntu terminal and connected to the underlying control system using USB through the Python serial port library.

[0052] The bottom control system receives the packaged recognition results and decodes them. The STM32 H7 chip is used as the main control of the control system to light up the indicator lights corresponding to the fruit type and whether the fruit is rotten, and intuitively display the recognition results to the user. The RM2006 and RM3508 motors are connected through CAN communication to control the sorting conveyor belt motor, which will be sent to the corresponding fruit type and whether the fruit is rotten.

[0053] Beneficial Effects

[0054] 1. The present invention is installed in the Ubuntu operating system and can be run by an industrial computer, thereby reducing the size of the present invention;

[0055] 2. The present invention uses the convolutional neural network model of the Resnet series as the recognition result, which greatly improves the recognition accuracy;

[0056] 3. The present invention can identify the type of fruit and whether the fruit is rotten, and at the same time control the conveyor belt to transfer the target fruit to the corresponding position, thereby improving the sorting efficiency and reducing the labor cost;

[0057] 4. The underlying control system of the present invention uses the STM32 series as the main control chip, which provides a strong guarantee for user interaction and motor control, and is also convenient for subsequent upgrades and maintenance.

[0058] The above disclosure is only a preferred embodiment of the fruit type and spoilage detection and sorting system based on deep learning of the present invention. Of course, this cannot be used to limit the scope of rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of the above embodiments and equivalent changes made according to the claims of the present invention still fall within the scope of the invention.

Claims

1. A fruit type and spoilage detection and sorting system based on deep learning, characterized in that: Including target recognition system and underlying control system: The target recognition system includes a camera module, a preprocessing module, a deep learning module and a first communication module. The camera module, the preprocessing module, the deep learning module and the first communication module are connected in sequence. The camera module is used to collect fruit image data in real time; The pre-processing module is used to adjust the resolution, brightness and saturation of the fruit image collected in real time; The deep learning module is used to perform model training and recognition on the fruit images collected in real time to obtain the type of fruit and whether it is rotten; The first communication module is used to transmit the recognition result to the underlying control system.

2. A fruit type and spoilage detection and sorting system based on deep learning as claimed in claim 1, characterized in that: The deep learning module includes a video framing module, a Resnet module, a fully connected layer module and a classification data module. The video framing module, the Resnet module, the fully connected layer module and the classification data module are connected in sequence. The video framing module is used to divide the real-time collected video stream data into multiple frames of picture data; The Resnet module is used as the core of the convolutional neural network recognition of the system; The fully connected layer module is used to calculate the probability of each fruit type and whether the fruit is rotten in the image data; The classification data module is used to find the value with the highest probability to determine the corresponding fruit type and whether the fruit is rotten.

3. A fruit type and spoilage detection and sorting system based on deep learning as claimed in claim 1, characterized in that: The bottom control system includes a second communication module, a data receiving module, a main control module, an indicator light module, a motor drive module and a power module. The second communication module, the data receiving module, the main control module, the indicator light module and the motor drive module are connected to the power module respectively. The second communication module, the data receiving module and the main control module are connected in sequence. The motor drive module and the indicator light module are connected to the main control module in sequence. The second communication module is used to transmit information data of the first communication module in the target identification system; The data receiving module is used to decode and process the character string data in the target recognition system and convert the character string data into a corresponding state to the main control module. The main control module is used to process all data in the underlying control system; The indicator light module is used to light up the indicator lights corresponding to the fruit type and whether the fruit is rotten, so as to intuitively display the recognition result to the user; The motor drive module is used to control the conveyor belt motor for sorting, and deliver the fruit to the position corresponding to the type of fruit and whether it is rotten; The power supply module stops supplying power to the second communication module, the data receiving module, the main control module, the indicator light module and the motor driving module.

Images