Small apple intelligent sorting system and method based on three-view image and weight information

By adopting an intelligent sorting method combining three-view image and weight information in the Apple sorting system, using improved deep learning models and decision-making layer fusion technology, the problems of low efficiency and inability to detect in real time by traditional Apple sorting equipment are solved, and the problem of inefficiency and inability to detect in real time is realized. Efficient and accurate apple sorting in orchard fields reduces costs.

CN119187034APending Publication Date: 2024-12-27NANJING AGRICULTURAL UNIVERSITY
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202411625280.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-11-14
Publication Date
2024-12-27

AI Technical Summary

Technical Problem

Traditional apple sorting equipment is inefficient and labor-intensive, and it is impossible to conduct quality grading inspections of apples in the orchard in real time, increasing transportation and storage costs.

Method used

Design a small Apple intelligent sorting system based on three-view image and weight information. Using the improved MobileNetV2 network model combined with the ECA attention mechanism, the weight and surface defect information of Apple is detected through the weight sensor and the three-view image acquisition device, and through The decision-making layer integrates multimodal data for intelligent sorting.

Benefits of technology

It realizes apple surface defect detection and multi-index classification, improves sorting accuracy and efficiency, reduces equipment costs, and can be installed on field working vehicles to achieve efficient, accurate and intelligent sorting in orchard fields.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119187034A_ABST
    Figure CN119187034A_ABST
Patent Text Reader

Abstract

The invention discloses a small apple intelligent sorting system and method based on three-view images and weight information, and relates to the field of apple quality grading. Comprising a feeding module, a detection module, a transmission module, a sorting module, an electrical control module and a boxing module which are sequentially connected. The detection module comprises a weight sensor and a three-view image acquisition device and is used for acquiring weight information and three-view image information of an apple to judge the grade of the apple; the three-view-angle image acquisition device comprises two plane mirrors and an RGB camera, the two plane mirrors are installed on the two sides of the objective table respectively, the included angle between each mirror surface and the horizontal plane is 60 degrees, and image information of the top of the apple and image information of the side faces of the apple in the two mirror surfaces are shot when the RGB camera shoots images. According to the intelligent apple sorting system, weight information and surface defect information of apples can be detected, the apples are intelligently sorted, meanwhile, the system is of a small structure and can be installed on an intelligent picking vehicle, and rapid and intelligent sorting operation on the apples in orchard fields is achieved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of apple quality grading, and more specifically to a small-scale apple intelligent sorting system and method based on three-viewing angle images and weight information. Background Art

[0002] my country is the world's largest apple producer and consumer, accounting for about 53.66% of the world's total apple production and about 52.99% of the world's total consumption. However, my country's apple industry has relatively low competitiveness in international trade and occupies a small market share. The reason is that my country's apple industry has weak post-harvest commercial processing capabilities, so it does not have strong market core competitiveness. Apple post-harvest processing is a repetitive and high-intensity job, which mainly includes online, cleaning, waxing, air-drying, sorting and other processes. Among them, sorting apples by quality is the key technology of apple post-harvest processing and the key production link to increase the added value of apple products. The research and development of apple sorting machines is of great significance to improving the level of apple post-harvest processing and promoting the international competitiveness of my country's apple industry. The traditional apple sorting method is inefficient and has high labor costs, which greatly affects the economic benefits of apple growers. In addition, some existing apple sorting equipment also has many limitations. At present, small apple sorting machines often have a relatively simple grading index, and only grade apples by weight or size, and cannot detect surface defects of apples. Although some large-scale intelligent apple grading lines can achieve multi-index grading, they are often large in structure, expensive, and have limited application scenarios. They cannot achieve real-time grading in orchards, and need to transport apples to special sorting line plants for sorting operations, which will further increase the transportation cost and storage difficulty of apples. Therefore, how to solve the above technical problems urgently requires further in-depth research by those skilled in the art. Summary of the invention

[0003] In view of this, for the on-site sorting task in the apple orchard, the present invention discloses a small-scale intelligent apple sorting system and method based on three-view images and weight information, which can detect the weight information and surface defect information of apples, and use this information to perform intelligent sorting of apples. At the same time, the system has a relatively small structure and can be installed on an intelligent picking vehicle to achieve rapid intelligent sorting of apples in the orchard field. The research and development of a small-scale intelligent apple sorting machine based on three-view images and weight information has a positive role in enriching the types of intelligent apple sorting equipment in my country, improving the intelligence level of sorting equipment and sorting efficiency, and is of great significance to promoting the development of my country's apple industry.

[0004] In order to achieve the above object, the present invention adopts the following technical solution:

[0005] A small-scale intelligent apple sorting system based on three-view images and weight information, comprising a feeding module, a detection module, a transmission module, a sorting module, an electrical control module and a packing module connected in sequence;

[0006] The detection module includes a weight sensor and a three-viewing angle image acquisition device, which is used to collect the weight information and three-viewing angle image information of the apple to determine the grade of the apple;

[0007] The three-view image acquisition device includes two plane mirrors and an RGB camera. The two plane mirrors are respectively installed on both sides of the stage. The angle between the mirror surface and the horizontal plane is 60°. When the RGB camera takes an image, it can capture the image information of the top of the apple and the image information of the side of the apple in the two mirror surfaces, thereby obtaining the image information of the apple surface at three different viewing angles.

[0008] Optionally, the three-view image acquisition device also includes a dark box and a light source, the RGB camera is located on the inner wall of the dark box frame, the light source is set at the top center of the dark box frame, and the dark box is used to reduce interference from external stray light.

[0009] Optionally, the detection module is improved by adding an ECA lightweight attention mechanism on the basis of the MobileNetV2 model. The ECA attention mechanism first performs global average pooling on each channel of the input feature map to extract global context information, and then performs one-dimensional convolution. The output of GAP is processed using one-dimensional convolution to learn the dependency between channels, wherein the kernel of the one-dimensional convolution is determined using formula (1):

[0010]

[0011] Where: C is the number of channels, γ is a hyperparameter, and σ is the rounding up operation.

[0012] Optionally, the improved MobileNetV2 network model consists of an initial convolutional layer, a series of inverted residual blocks and a tail convolutional layer, each inverted residual block contains a depth-separable convolution, and the convolutional layer and fully connected layer of the last part of the improved MobileNetV2 network model are used to generate the final classification result.

[0013] Optionally, after obtaining the grading results based on the weight of the apples and the grading results based on the image information, the detection module uses a decision layer fusion method to perform multimodal data fusion on the apple grade information.

[0014] Optionally, the sorting module is provided with different sorting areas, and different sorting areas correspond to different grades of fruits; each sorting area is equipped with an electromagnetic pushing device, and when the apples enter the corresponding sorting area, the electromagnetic pushing device is activated to cause the fruit tray to tip over, and the apples fall from the fruit tray into the packing slide of the packing module, completing the sorting and packing.

[0015] Optionally, an electrical control module is also included to integrate the electrical control part of the whole machine.

[0016] A small apple intelligent sorting method based on three-view images and weight information comprises the following steps:

[0017] Weigh the weight data of each apple, analyze the weight data, and determine the apple grade corresponding to the weight data;

[0018] The camera is installed above the apple, and three-view images of the apple are collected by installing plane mirrors on both sides of the collection area, and the corresponding apple grade is determined based on the three-view images;

[0019] The final decision is made by integrating the grading results of weight information and image information, with the inferior grade information being used as the final grade of the apple.

[0020] Optionally, it also includes converting the three-view images of the apple into a three-dimensional point cloud image, preprocessing the three-dimensional point cloud image, removing irrelevant point cloud information and outliers, and after chessboard calibration, selecting the rotation matrix and translation matrix of one perspective as the reference perspective, using coordinate transformation to unify the coordinates of another three-dimensional point cloud to the reference coordinate system, and then using the iterative closest point algorithm (ICP) algorithm to align the point clouds.

[0021] Optionally, the collected apple weight, appearance color, skin texture, and morphological features are fused and decided using a BP neural network to obtain the apple grade.

[0022] Through the above technical solutions, it can be known that compared with the prior art, the present invention provides a small-scale apple intelligent sorting system and method based on three-view images and weight information, and the beneficial effects are as follows: First, the miniaturized apple intelligent sorting machine designed by the present invention proposes an apple appearance quality detection scheme based on three-view images, and adopts a shooting method in which a single camera is matched with two plane mirrors, so that the image captured by the present invention using only one camera contains the information of three different perspectives of the apple, and the information of the three perspectives is in one image. Compared with the existing solution of using a single camera for apple surface quality detection, the detection range of the apple surface detected by the present invention is greatly increased, and the detection result is more accurate and more reliable. Compared with the existing solution of collecting multiple surfaces of apples through multiple cameras, the number of cameras is reduced, the equipment cost is effectively reduced, and the image processing is more convenient, making the training of the image classification model easier, the detection module structure is more compact, and the detection processing efficiency is higher. Secondly, the present invention combines the surface defect situation of the apple and the weight information for fusion decision-making, further improves the accuracy and reliability of the existing apple grading system, and realizes the apple grading decision of multi-classification index fusion. Finally, the present invention greatly reduces the space occupied by the device by designing and optimizing the mechanical structure, compared with the existing intelligent apple sorting line, and can be installed on field operation vehicles to achieve field online sorting in conjunction with manual picking or intelligent apple picking equipment. The present invention solves the problem of efficient, accurate and intelligent sorting of apples in orchards, which is of great significance to promoting the automation, informatization and intelligence of fruit grading in my country, and is of great significance to enhancing the commercial value of fruit crops in my country. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0024] Figure 1 This is the overall structure diagram of the small apple intelligent sorting system;

[0025] Figure 2 It is the system hardware relationship diagram;

[0026] Figure 3 It is the system workflow diagram;

[0027] Figure 4 This is the structure diagram of the detection module;

[0028] Figure 5 To improve the MobileNetV2 network model structure diagram;

[0029] Figure 6 It is a system decision-making process diagram;

[0030] Among them: 1-first collecting conveyor belt, 2-second collecting conveyor belt, 3-loading conveyor belt, 4-fruit tray, 5-detection module, 6-electrical control module, 7-rotating conveyor belt, 8-packing ramp. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] The embodiment of the present invention discloses a small-scale intelligent apple sorting system based on three-view images and weight information, comprising a feeding module, a detection module 5, a transmission module, a sorting module and a packing module connected in sequence;

[0033] The detection module 5 includes a weight sensor and a three-view image acquisition device, which is used to determine the grade of the apple based on the weight of the apple and the three-view image;

[0034] The three-view image acquisition device includes two plane mirrors and an RGB camera. The two plane mirrors are respectively installed on both sides of the stage, with an angle of 60° with the horizontal plane. When the RGB camera takes an image, it can capture the image information of the top of the apple and the side image information of the apple in the two mirror surfaces, thereby obtaining the surface image information of the apple at three different viewing angles.

[0035] like Figure 1 As shown, the small-sized intelligent apple sorting machine of the present invention has a maximum length of 2.5 meters and a maximum width of 1.7 meters. It is divided into six parts: a loading module, a detection module 5, a transmission module, a sorting module, an electrical control module 6, and a packing module. The packing module includes a packing ramp 8. The hardware relationship diagram thereof is shown in FIG. Figure 2. The apple loading module designed by the present invention is used for collecting apples after picking and ensuring that the apples enter the subsequent detection module 5 individually and orderly. The main body of the device is composed of three conveyor belts. The first collecting conveyor belt 1 and the second collecting conveyor belt 2 installed on both sides are 90 cm high and 170 cm wide in total. They are used for collecting apples after picking, and can realize the simultaneous collection of apples on both sides. The loading conveyor belt 3 located in the middle is 80 cm high and 50 cm long. It cooperates with the two paddles at its end to arrange the apples into a single row for orderly loading. The transmission module of the present invention is constructed by a flat rotary transmission bar and a plurality of fruit trays 4. The flat rotary fruit tray reduces the length of the machine and makes the space on both sides of the rotation available. The length from the first section to the tail end is 2 meters, and the rotation width is 60 cm. Each fruit tray can accommodate one apple to realize the transmission of apples. The mounting platform of the rotary conveyor belt 7 has a large area, so that the detection module 5 and the electrical control module 6 can be installed overlapping with the transmission module and the sorting module, saving space. The detection module 5 of the present invention is installed at the front end of the transmission module, followed by the feeding module, and is composed of a dark box, a weighing platform, a plane mirror, a camera, and a ring light source. The apple completes the information collection in this module. The electrical control module 6 is installed above the rotating inner circle of the transmission module, integrating the electrical control part of the whole machine. The sorting module is composed of four electromagnetic driving devices and a fruit tray. The electromagnetic driving device is installed below the rotary conveyor belt 7. One is installed on the same side of the detection module 5 and three are installed on the other side. After receiving the action command, the device will push the fruit tray, causing the fruit tray to overturn so that the apple slides out of the fruit tray to complete the sorting action. The fallen apples are packed along the packing slide. The packing module of the present invention is composed of a packing slide and four fruit boxes placed at the end of the packing slide. The four fruit boxes are used to collect grade A fruit, grade B fruit, grade C fruit and discarded fruit respectively. The bottom and sides of the fruit box are padded with soft sponge pads, which can effectively reduce the collision damage caused by the falling of apples during the packing process.

[0036] like Figure 3The main working process of this small apple intelligent sorting machine is divided into five stages: loading stage, apple weight information and image information detection stage, data processing and fusion decision stage, sorting operation stage, and apple packing stage. In the loading stage, the picked apples are placed in the collecting conveyor belts on both sides of the device, and the apples are collected on the loading conveyor belt located in the middle of the device. The conveyor belt arranges the apples in a single row through two inward-contracting paddles, and finally falls into the fruit plate in an orderly manner. Each fruit plate contains one apple, thereby completing the loading work. Then, in the apple weight information and image information detection stage, the apple and the detection module 5 complete the acquisition of weight information and image information, and transmit it to the electrical control module 6 through the signal line. The upper computer completes data processing and fusion decision-making, and outputs apple grade information. According to this, the apples entering the sorting area are sorted. The fruit plate conveyor belt is continuously driven. After the information detection is completed in the detection module 5, the apples are carried into the sorting area by the rotary conveyor belt 7. The apples are finally divided into four grades, namely, grade A fruit, grade B fruit, grade C fruit, and defective fruit. The present invention sets a sorting area on the same side of the detection module 5, corresponding to grade A fruit, and sets three sorting areas on the opposite side, corresponding to grade B fruit, grade C fruit and defective fruit, and each sorting area is equipped with an electromagnetic propulsion device. When the apples enter the corresponding sorting areas of grades A, B, and C, the electromagnetic propulsion device is activated to cause the fruit tray to tip over, and the apples fall from the fruit tray into the packing slide of the packing module, completing the sorting and packing. When the fruit tray enters the defective product area, the electromagnetic propulsion device always performs a tipping action on each fruit tray to ensure that each fruit tray completes the next round of loading in an idle state, and the tipped fruit tray is restored to an upright state by the restoration device before the next round of loading, and this cycle is repeated to achieve a complete sorting process for apples.

[0037] In order to realize the collection of three-view image information and weight information, the present invention designs the following Figure 4The detection module 5 shown is composed of a dark box, a weighing platform, a plane mirror, a camera, and a ring light source. The dark box is used to reduce the interference of external stray light. A high-precision weight sensor is provided under the weighing platform to place apples and obtain weight information. The measured analog signal is transmitted to the data acquisition card, which is then transmitted to the computer via the USB interface. The three-view image of the apple is obtained in the following way: the plane mirrors are installed on both sides of the platform, with an angle of 60° with the horizontal plane. The RGB camera is connected to the graphics workstation via a USB cable. The software trigger method is adopted. When the apple passes through the detection area, the camera located at the top of the detection module 5 takes pictures. The camera can directly take pictures of the front image of the apple parallel to the camera, and the side image of the apple obtained by the reflection in the plane mirrors on both sides will also be taken together. Therefore, the information of the complete surface of the apple is obtained from three perspectives through one image acquisition. The ring light source is fixed on the camera lens to provide a uniform and stable light source during image acquisition. The acquired image is processed by the classification model. The input of the model is the three-view image of the apple, and the output result is the apple grade.

[0038] For the classification model of image information, the present invention adopts deep learning method to train it, and the specific principle is as follows:

[0039] In order to solve the problem of multi-faceted image classification of apples, the present invention improves the MobileNetV2 model to obtain a more efficient model, which enhances the recognition ability of the model without adding too many parameters and calculations. The improved model is obtained by adding the ECA (Efficient Channel Attention) lightweight attention mechanism on the basis of the MobileNetV2 model. The ECA attention mechanism first performs global average pooling on each channel of the input feature map to extract global context information. This step generates a one-dimensional vector channel descriptor whose length is equal to the number of channels of the feature map. Then, a one-dimensional convolution is performed to process the output of GAP to learn the dependencies between channels. The kernel size of the one-dimensional convolution is adaptive and dynamically determined according to the number of channels of the feature map. The calculation of the kernel size is related to the number of channels of the feature map. Formula (1) can be used to determine the kernel size:

[0040]

[0041] Where: C is the number of channels, γ is a hyperparameter, and σ is the rounding up operation.

[0042] The ECA module is added to the end of each inverted residual block of MobileNetV2, specifically after the second PW convolution. Figure 5To improve the MobileNetV2 network model structure, it consists of an initial convolutional layer, a series of inverted residual blocks, and a tail convolutional layer. Each inverted residual block contains a depth-separable convolution. The convolutional layer and fully connected layer in the last part of the model are used to generate the final classification results. The modules of Bottleneck-s1 and Bottleneck-s2 refer to inverted residual structures with strides of 1 and 2 respectively, and n is the number of repetitions of the corresponding module.

[0043] In model training, the present invention divides the collected sample images into training set, validation set and test set in a ratio of 6:2:2, and then performs data enhancement on each set. The training set is used for model training; the validation set does not participate in training, and checks the state and convergence of the model during the training process, and adjusts the hyperparameters of the model; the test set also does not participate in training, and is used to test the model after the training is completed to evaluate its generalization ability.

[0044] In actual use, the camera transmits the captured apple image to the host computer through the data line, and the host computer preprocesses the image. Specifically, each image is adjusted to 224*224 pixels and the pixel value is normalized to between [0,1] to meet the input requirements of the model. After preprocessing, the apple image will be input into the trained model for inference. The highest probability result shown in the probability distribution output by the model is the final model prediction result and output. In the present invention, the model will output four possible apple grades, which are divided into grade A apples, grade B apples, grade C apples and defective products from good to bad.

[0045] This embodiment also discloses a small apple intelligent sorting method based on three-view images and weight information, including the following steps:

[0046] Weigh the weight data of each apple, analyze the weight data, and determine the apple grade corresponding to the weight data;

[0047] The camera is installed above the apple, and three-view images of the apple are collected by installing plane mirrors on both sides of the collection area, and the corresponding apple grade is determined based on the three-view images;

[0048] The final decision is made by integrating the grading results of weight information and image information, with the inferior grade information being used as the final grade of the apple.

[0049] Furthermore, it also includes converting the three-view images of the apple into a three-dimensional point cloud map, pre-processing the three-dimensional point cloud map, removing irrelevant point cloud information and outliers, and after chessboard calibration, selecting the rotation matrix and translation matrix of one perspective as the reference perspective, using coordinate transformation to unify the coordinates of another three-dimensional point cloud to the reference coordinate system, and then using the iterative closest point algorithm (ICP) algorithm to align the point cloud.

[0050] Furthermore, the collected apple weight, appearance color, skin texture, and morphological characteristics are fused and decided using a BP neural network to obtain the apple grade.

[0051] More specifically, in order to combine the surface defects of apples with weight information to make apple grading decisions, after obtaining the grading results based on the weight of the apples and the grading results based on the image information, the present invention uses a decision-making layer fusion method to perform multi-modal data fusion on the apple grade information to make up for the shortcomings of single-modal data and enhance the comprehensive capabilities of the system. By analyzing the three-view image information and weight information, the present invention fuses the decision results of the two detection methods based on the decision layer, classifies the image information using the corresponding model, classifies the weight information according to a given grade interval, fuses the classification results of the two detection methods, and outputs the final result, such as Figure 6 The figure shows the fusion decision process diagram. The specific process is as follows:

[0052] The first step is weight information data collection and processing: the weight sensor is installed under the weighing platform to weigh the weight data of each apple, analyze the weight data, and compare the size to which level of weight range it belongs. The final output result is divided into grade A apples, grade B apples, grade C apples, and defective products from good to bad.

[0053] The second step is image data acquisition and processing: the camera is installed above the apple, and three-view images of the apple are collected by adding plane mirrors on both sides of the acquisition area. The image data is analyzed through the image classification model. The final output results are divided into grade A apples, grade B apples, grade C apples, and defective products from good to bad.

[0054] The third step is fusion decision: make the final decision based on the grading results of weight information and image information. The specific approach is to compare the inferior grading results of the two types of information and use the inferior grade information as the final grade of the apple.

[0055] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0056] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A small-scale intelligent apple sorting system based on three-view images and weight information, characterized in that: It includes a feeding module, a detection module, a transmission module, a sorting module, an electrical control module and a packing module which are connected in sequence; The detection module includes a weight sensor and a three-viewing angle image acquisition device, which is used to collect the weight information and three-viewing angle image information of the apple to determine the grade of the apple; The three-view image acquisition device includes two plane mirrors and an RGB camera. The plane mirrors are respectively installed on both sides of the stage. The angle between the mirror surface and the horizontal plane is 60°. When the RGB camera takes an image, it captures the image information of the top of the apple and the side image information of the apple in the two mirror surfaces, thereby obtaining the surface image information of the apple at three different viewing angles.

2. According to claim 1, a small-sized apple intelligent sorting system based on three-view images and weight information is characterized in that: The three-view image acquisition device also includes a dark box and a light source. The RGB camera is located on the inner wall of the dark box frame, and the light source is arranged at the top center of the dark box frame. The dark box is used to reduce the interference of external stray light.

3. According to claim 1, a small-sized apple intelligent sorting system based on three-view images and weight information is characterized in that: The detection module improves the MobileNetV2 network model by adding the ECA lightweight attention mechanism on the basis of the MobileNetV2 model. The ECA attention mechanism first performs global average pooling on each channel of the input feature map to extract global context information, and then performs one-dimensional convolution. The output of GAP is processed using one-dimensional convolution to learn the dependency between channels, where the kernel of the one-dimensional convolution is determined using formula (1): Where: C is the number of channels, γ is a hyperparameter, and σ is the rounding up operation.

4. The small-sized apple intelligent sorting system based on three-view images and weight information according to claim 3, characterized in that: The improved MobileNetV2 network model consists of an initial convolutional layer, a series of inverted residual blocks and a tail convolutional layer, each inverted residual block contains a depth-separable convolution, and the convolutional layer and the fully connected layer in the last part of the improved MobileNetV2 network model are used to generate the final classification result.

5. The small-sized apple intelligent sorting system based on three-view images and weight information according to claim 1, characterized in that: The detection module uses a decision layer fusion method to perform multimodal data fusion on the apple grade information based on the grading results of the apple weight and the grading results based on the image information.

6. The small-sized apple intelligent sorting system based on three-view images and weight information according to claim 1, characterized in that: The sorting module is provided with different sorting areas, and different sorting areas correspond to different grades of fruits; each sorting area is equipped with an electromagnetic pushing device. When the apples enter the corresponding sorting area, the electromagnetic pushing device is activated to cause the fruit tray to tip over, and the apples fall from the fruit tray into the packing slide of the packing module, completing the sorting and packing.

7. The small-sized apple intelligent sorting system based on three-view images and weight information according to claim 1, characterized in that: It also includes an electrical control module for integrating the electrical control part of the whole machine.

8. An intelligent sorting method for small apples based on three-view images and weight information, characterized in that: The following steps are involved: Weigh the weight data of each apple, analyze the weight data, and determine the apple grade corresponding to the weight data; The camera is installed above the apple, and three-view images of the apple are collected by installing plane mirrors on both sides of the collection area, and the corresponding apple grade is determined based on the three-view images; The final decision is made by integrating the grading results of weight information and image information, with the inferior grade information being used as the final grade of the apple.

9. The method for intelligent sorting of small apples based on three-view images and weight information according to claim 8, characterized in that: It also includes converting the three-view images of the apple into a three-dimensional point cloud, preprocessing the three-dimensional point cloud, removing irrelevant point cloud information and outliers, and selecting the rotation matrix and translation matrix of one perspective as the reference perspective after chessboard calibration. Coordinate transformation is used to unify the coordinates of another three-dimensional point cloud to the reference coordinate system, and then the point cloud is registered using the iterative closest point algorithm (ICP) algorithm.

10. The method for intelligent sorting of small apples based on three-view images and weight information according to claim 8, characterized in that: The collected apple weight, appearance color, skin texture and morphological characteristics are fused and decided using BP neural network to obtain the apple grade.

Citation Information

Patent Citations

  • Pomelo posture and quality detection method and device and storage medium

    CN110586491A

  • Shooting method for three-dimensional strain deformation measurement

    CN111336942A

  • Apple grading assembly line system and method based on double RGB-D cameras

    CN117299614A

  • Pomelo fruit posture adjusting device for nondestructive testing of quality assembly line

    CN210676005U

  • Apparatus for manufacturing display unit

    KR1020200137912A