A multi-spectral imaging online identification system for slight damage on fruit surface

Abstract

The application discloses a kind of online identification system of the multispectral imaging of slight damage of fruit surface, including fruit tray and track, conveying mechanism, material sorting mechanism, detection mechanism, classification mechanism, collection mechanism;Material sorting mechanism includes two guide panels, detection mechanism includes proximity sensor A and multispectral camera, multispectral camera is customized according to damage characteristic wavelength, the information transmission of the fruit to be measured detected by proximity sensor A is given to control unit, and then control active light source work and the running speed and tension of conveying belt, multispectral camera obtains the spectral image of the fruit to be measured, and damage image recognition is carried out by YOLOv8 model;Classification mechanism includes a pair of electromagnet array system, electromagnet array system rear side is equipped with proximity sensor B, electromagnet array system can receive the instruction of control unit and accurately magnetic field force is applied to fruit tray, and slightly damaged fruit and intact fruit are moved to different sides.The application can quickly and accurately detect the slight damage of various fruits.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of fruit sorting and grading, and particularly relates to a multi-spectral imaging online identification system for slight damage on the surface of fruits. BACKGROUND

[0002] Fruits contain rich ingredients such as minerals, vitamins, fatty acids and dietary fiber, and have a sweet and sour taste, which are favored by consumers. However, during the fruit harvesting process, the fruit is subjected to external physical factors such as squeezing, collision, vibration and friction during picking, packaging, storage and transportation, which can cause physical damage such as slight indentation and surface damage. These early damages are difficult to identify by the naked eye at the initial stage, and manual inspection can further increase the likelihood of physical damage and consume a large amount of manpower during the detection process. When the damaged fruit is packaged and enters the circulation supply chain, the damaged part is easily infected by external bacteria, causing fruit rotting, and even spreading to other healthy fruits, resulting in a large number of products being unable to be sold normally, causing economic losses and affecting the reputation and image of the brand, and having a long-term negative impact on the industry. Therefore, it is of great significance for the development of the fruit industry to quickly and non-destructively identify early damage on the surface of fruits.

[0003] When the fruit is damaged by external physical factors, the peel and flesh tissue will change in physiology and physics, such as the diffusion of epidermal cell tissue fluid, which changes the optical properties of the fruit surface. Compared with the traditional manual detection method, optical detection technology has the advantages of high speed, non-destructive and low labor, and is an important means for online detection of the external quality of fruits and vegetables. Using conventional machine vision systems can effectively identify severe damage to fruits such as rotting, scarring and squeezing, but ordinary machine vision is difficult to detect early and slight damage to fruits. High-spectral imaging technology can clearly detect subtle damage that is difficult to distinguish by the naked eye and RBG images, and can also provide rich spatial and spectral information, which helps to better understand the damage to the fruit. However, high-spectral imaging technology has the disadvantages of high equipment cost, complex data processing and limited detection speed, which is not suitable for real-time fruit damage detection in industry, and needs to combine the advantages of multi-spectral imaging systems that can be used for rapid detection of large quantities of fruits on industrial production lines to meet the actual production needs. SUMMARY

[0004] In view of the deficiencies in the prior art, the present application provides a multi-spectral imaging online identification system for slight damage on the surface of fruits.

[0005] The present application achieves the above technical purpose by the following technical means.

[0006] A multi-spectral imaging online identification system for slight damage on the surface of fruits comprises:

[0007] A plurality of fruit trays and tracks, each fruit tray and track comprising a fruit tray and a track, the fruit tray being located on the track and being able to move horizontally along the track;

[0008] A conveying mechanism comprising a support one, a support three and a conveying chain set, the conveying chain set comprising a conveying belt and a motor, a plurality of tracks being fixed at equal intervals on the conveying belt;

[0009] A sorting mechanism, a detection mechanism, a classification mechanism and a collection mechanism are sequentially arranged along the movement direction of the conveying belt;

[0010] The sorting mechanism comprises guide panels arranged on both sides of the conveying belt, which are used to guide the transported fruits to be tested to the center line position of the conveying belt;

[0011] The detection mechanism comprises a proximity sensor A, a multi-spectral camera and an active light source, the information of the fruits to be tested detected by the proximity sensor A is transmitted to the control unit, the control unit controls the operation of the active light source, and controls the running speed and tension of the conveying belt according to the signal of the proximity sensor A, so as to ensure that each fruit to be tested can be captured by the multi-spectral camera in the best state; the multi-spectral camera is a multi-lens multi-spectral camera customized according to the damage characteristic wavelength obtained by screening, different filters are embedded in a plurality of lenses, and each lens is equipped with a CMOS image sensor; the multi-spectral camera acquires the spectral image of the fruit to be tested and sends it to the detection terminal, and a YOLOv8 model in the detection terminal performs damage image recognition;

[0012] The classification mechanism comprises a set of electromagnet array systems arranged on both sides of the conveying belt respectively, and a proximity sensor B is arranged on the rear side of the electromagnet array system; the electromagnet array system is integrated with a controller, which can receive the instructions of the control unit and apply precise magnetic field force to the fruit tray, so as to move the slightly damaged fruits and the intact fruits to different sides;

[0013] The collection mechanism is specifically two slides.

[0014] In the above technical solution, the rotating shafts at both ends of the conveying belt are fixed on the support one and the support three respectively, one of the rotating shafts is driven by the motor, and the operation of the motor is controlled by the control unit; a pair of baffles is installed between the support one and the support three, the sorting mechanism and the classification mechanism are arranged on the baffles, the detection mechanism is arranged on the support two, the support two is located at the middle position of the conveying mechanism, and the collection mechanism is arranged on the support three.

[0015] In the above technical solution, the working principle of the electromagnet array system is that in the initial stage, the electromagnet rapidly establishes sufficient magnetic field force by passing a large current, so as to ensure that the fruit tray is stably attracted and moved; then, the current is gradually reduced in a proportional manner, so as to slow down the acceleration of the fruit tray.

[0016] In the technical solution, the fruit damage characteristic wavelength is screened by the following steps: collecting hyperspectral image data of healthy fruits in sequence according to the number order, extracting average reflectivity spectrum of a healthy area of each fruit after two times of hyperspectral image collection, slightly damaging the healthy fruits, collecting hyperspectral image data of the damaged fruits, extracting average reflectivity spectrum of a damaged area of each fruit, and screening the fruit damage characteristic wavelength by using the average reflectivity spectrum data.

[0017] In the technical solution, the fruit damage characteristic wavelength is screened by the following steps: collecting hyperspectral image data of healthy fruits in sequence according to the number order, extracting average reflectivity spectrum of a healthy area of each fruit after two times of hyperspectral image collection, slightly damaging the healthy fruits, collecting hyperspectral image data of the damaged fruits, extracting average reflectivity spectrum of a damaged area of each fruit, and screening the fruit damage characteristic wavelength by using the average reflectivity spectrum data.

[0018] In the technical solution, the fruit damage characteristic wavelength is screened by the following steps: collecting hyperspectral image data of healthy fruits in sequence according to the number order, extracting average reflectivity spectrum of a healthy area of each fruit after two times of hyperspectral image collection, slightly damaging the healthy fruits, collecting hyperspectral image data of the damaged fruits, extracting average reflectivity spectrum of a damaged area of each fruit, and screening the fruit damage characteristic wavelength by using the average reflectivity spectrum data.

[0019] In the technical solution, the fruit damage characteristic wavelength is screened by the following steps: collecting hyperspectral image data of healthy fruits in sequence according to the number order, extracting average reflectivity spectrum of a healthy area of each fruit after two times of hyperspectral image collection, slightly damaging the healthy fruits, collecting hyperspectral image data of the damaged fruits, extracting average reflectivity spectrum of a damaged area of each fruit, and screening the fruit damage characteristic wavelength by using the average reflectivity spectrum data.

[0020] In the technical solution, the fruit damage characteristic wavelength is screened by the following steps: collecting hyperspectral image data of healthy fruits in sequence according to the number order, extracting average reflectivity spectrum of a healthy area of each fruit after two times of hyperspectral image collection, slightly damaging the healthy fruits, collecting hyperspectral image data of the damaged fruits, extracting average reflectivity spectrum of a damaged area of each fruit, and screening the fruit damage characteristic wavelength by using the average reflectivity spectrum data.

[0020] In the technical solution, the fruit damage characteristic wavelength is screened by the following steps: collecting hyperspectral image data of healthy fruits in sequence according to the number order, extracting average reflectivity spectrum of a healthy area of each fruit after two times of hyperspectral image collection, slightly damaging the healthy fruits, collecting hyperspectral image data of the damaged fruits, extracting average reflectivity spectrum of a damaged area of each fruit, and screening the fruit damage characteristic wavelength by using the average reflectivity spectrum data.

[0021] In a certain wave band, for the average reflectivity of a single sample, if it is greater than the average reflectivity of the whole, it is subtracted by twice the difference between the value and the average reflectivity of the whole, if it is less than the average reflectivity of the whole, it is added by twice the difference between the value and the average reflectivity of the whole, and if it is exactly equal to the average reflectivity of the whole, it is kept unchanged.

[0022] In the technical solution, the clustering random frog algorithm obtains the spectrum of the wave band with the top three SR values, and specifically:

[0023] The values of c, epsilon and k are set; in the first step, K average reflectivity spectra are randomly selected as initial centroids, the distance between the average reflectivity spectrum of each sample (fruit) and each initial centroid is calculated, each sample is assigned to the group in which the nearest initial centroid is located, in the second step, the centroid of each group (the centroid is the average of the average reflectivity of all samples in the group) is calculated, and then the average reflectivity spectrum of each sample is re-assigned to the group in which the centroid with the nearest distance to the average reflectivity spectrum of the sample is located, the centroid of the current group is calculated again and re-assigned, and the second step is iterated until the iteration number reaches the preset value c; then, the algorithm randomly selects an initial subset V0 containing Q variables as a starting point, in each iteration, the algorithm randomly generates a number Q* from a normal distribution Norm (Q, hQ), which represents the number of variables in the candidate variable subset V*; then, the initial subset V0 is updated according to the relationship between Q and Q*; then, a random number between 0 and 1 is generated, if the random number is greater than or equal to the preset threshold, V* is accepted as the new feature subset V N+1 , otherwise, V* is rejected and the variable subset of the current iteration is retained; the algorithm also calculates the change of the classification accuracy of the adjacent two iterations, if the change is less than the preset value epsilon and the iteration number is greater than the preset value k, the iteration is stopped, otherwise, the iteration is still performed until the iteration number reaches the preset maximum iteration number; after the iteration is completed, the selection probabilities SR values of all variables are arranged in descending order, and the top three variables with the SR values are the damage feature wavelengths.

[0024] In the technical solution, the initial subset V0 is updated according to the relationship between Q and Q*, and specifically:

[0025] If Q is equal to Q*, the classification accuracy of V* and V0 is calculated and compared, if the classification accuracy of V* is greater than V0, V* is replaced by V0, if the classification accuracy of V* is less than or equal to V0, no replacement is performed;

[0026] If Q is less than Q*, Q*-Q variables related to the minimum regression coefficient are deleted from V0;

[0027] If Q is greater than Q*, select Q-Q* variables in V* associated with the largest regression coefficients and add to V0.

[0028] In the technical solution, two groups of damage characteristic wavelengths screened out by the sub-window mirror permutation analysis algorithm and the clustering random frog algorithm are fused into images, and N characteristic wavelength images of fruit damage are obtained, which are divided into a training set and a verification set of the YOLOv8 model according to a ratio of 7:3, and the ratio of characteristic wavelength images of damaged fruits to characteristic wavelength images of healthy fruits in the verification set is 1:1.

[0029] Compared with the prior art, the beneficial effects of the present application are:

[0030] (1) The present application uses the YOLOv8 model for damage image recognition, and the training set and the verification set of the YOLOv8 model are images of two groups of damage characteristic wavelengths screened out by the sub-window mirror permutation analysis algorithm and the clustering random frog algorithm respectively; the present application is suitable for slight damage detection of various types of fruits and can clearly and accurately detect slight damage that cannot be distinguished by the naked eye and an RGB image; the present application uses hyperspectral imaging technology with the advantages of high spectral resolution and suitable machine learning algorithms to screen damage characteristic wavelengths and customize a multi-lens multi-spectral camera; the present application applies the multi-spectral camera to real-time detection of slight damage of fruits on an industrial production line, which can meet the requirements of fast, accurate and low-cost detection in business.

[0031] (2) The system of the present application can automatically and quickly complete detection and accurate classification of whether the fruits have slight damage without causing any additional damage to the fragile fruits; this feature is particularly important in practical applications, which not only effectively improves the efficiency of fruit quality detection, but also avoids secondary damage to the fruits caused by improper human operation; in addition, the system has wide applicability to most types of fragile fruits, enhancing its practicality in actual production environment. It is worth mentioning that the hardware devices used in the system have relatively low cost, and this economic advantage makes the system have high cost performance and great market potential, which can be widely promoted and applied, thereby bringing substantial convenience and benefits to the fruit industry.

[0032] (3) Compared with other related inventions, the significant advantage of the present application lies in its high practicability and wide applicability, which makes it easier for various enterprises to adopt and widely apply. Not only suitable for various types of perishable fruits, but also stable operation in different environments, which greatly enhances its practical value. The present application has been strictly scientifically verified to ensure the accuracy and reliability of detection and classification, providing strong quality guarantee for enterprises. Secondly, the high processing capacity of the system is one of its outstanding advantages, which can quickly complete the detection and classification task of a large number of fruits, effectively improving the work efficiency. The present application adopts advanced algorithm and hardware design to ensure the efficient and stable operation of the system, and also brings greater competitive advantage to enterprises. BRIEF DESCRIPTION OF DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0034] Figure 1 Schematic diagram for screening characteristic wavelength by using clustering random frog algorithm according to the present application;

[0035] Figure 2 Schematic diagram for screening characteristic wavelength by using sub-window mirror image permutation analysis algorithm according to the present application;

[0036] Figure 3 Structural schematic diagram of the multi-spectral imaging online identification system for slight damage on fruit surface according to the present application;

[0037] Figure 4 Schematic diagram of the material sorting mechanism according to the present application;

[0038] Figure 5 Side view of the electromagnet array system according to the present application;

[0039] Figure 6 Side view of the detection mechanism for identifying fruit damage according to the present application;

[0040] Figure 7 Flow chart of the multi-spectral imaging identification method for slight damage on fruit surface according to the present application;

[0041] In the figure, 101 - fruit tray and track, 102 - conveying mechanism, 103 - sorting mechanism, 104 - detection mechanism, 105 - classification mechanism, 106 - collection mechanism, 107 - guide panel, 108 - proximity sensor A, 109 - multi-spectral camera, 110 - bracket one, 111 - active light source, 112 - bracket three, 113 - bracket two, 114 - proximity sensor B. DETAILED DESCRIPTION

[0042] It should be noted that the following detailed description is exemplary in nature and is intended to provide further description of the application. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0043] It is to be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments according to the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, steps, operations, elements, components, and / or groups thereof, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof.

[0044] Example 1

[0045] In order to find the characteristic wavelength of the damaged fruit, the hyperspectral image data of the fruit needs to be collected using a hyperspectral imaging system. First, the prepared healthy fruit is collected in order according to the number, and the hyperspectral image data of each fruit is collected twice. After the image is corrected for black and white, the average reflectivity spectrum of the healthy area of each fruit is extracted. Next, the above-mentioned healthy fruit is artificially damaged, and the same process of collecting hyperspectral image data, correcting black and white, and extracting the average reflectivity spectrum of the damaged area of each fruit is performed twice. The average reflectivity spectrum data is used to screen the characteristic wavelength of the fruit damage. Specifically:

[0046] After the parameters of the hyperspectral imaging system are set, the camera lens cover is covered, the hyperspectral image data with a reflectance of 0% is collected, and is recorded as dark reference R dark , and the hyperspectral image data of the light source fully reflected by the polytetrafluoroethylene reflector with a reflectance of 99% is collected, and is recorded as white reference R white .

[0047] After the fruit hyperspectral image data is collected, the fruit hyperspectral image data is corrected for black and white using the following formula according to the obtained dark reference and white reference.

[0048]

[0049] Wherein, pxy (i) refers to the reflectance of the fruit at spatial coordinates (x, y), after correction, at spectral wavelength i, R xy (i) refers to the original intensity of the fruit spectrum at spatial coordinates (x, y), at spectral wavelength i, I

[0050] Then, the average reflectance spectrum of the damaged area of the fruit is extracted, and the average reflectance spectrum corresponding to the healthy area is extracted.

[0051] The average reflectance spectrum of the healthy fruit is analyzed using the sub-window mirror replacement analysis algorithm and the clustering random jumping frog algorithm, and the average reflectance spectrum of the damaged fruit is also analyzed using the sub-window mirror replacement analysis algorithm and the clustering random jumping frog algorithm. The sub-window mirror replacement analysis algorithm obtains several wavelength spectra with high COSS values, and the clustering random jumping frog algorithm obtains several wavelength spectra with large SR values, which are respectively selected as the damage characteristic wavelengths of the two algorithms.

[0052] As Figure 2As shown, the sub-window mirror permutation analysis algorithm analyzes the average reflectance spectrum of the fruit, and several damage characteristic wavelengths are screened out in the near-infrared band. The number of spectral variables j and the number of repetitions N are set. It is assumed that, in the sub-data set obtained by sampling the average reflectance spectrum of the fruit in the first step of Monte Carlo, there are J spectral sub-data sets containing the number of spectral variables j. In the second step of establishing the PLS-LDA classification model, there are J PLS-LDA classification models containing the number of spectral variables j. At this time, the prediction error of the prediction set of the J models is called the normal prediction error (NPE). In addition, the spectral matrix of the prediction set of the J models is subjected to mirror permutation, which is specifically divided into: under a certain waveband, for the average reflectance of a single sample, if it is greater than the average reflectance of the population, it is subtracted by twice the difference between the average reflectance of the single sample at the waveband and the average reflectance of the population, if it is less than the average reflectance of the population, it is added by twice the difference between the average reflectance of the single sample at the waveband and the average reflectance of the population, and if it is exactly equal to the average reflectance of the population, it remains unchanged; a certain variable among the j spectral variables is selected as a permutation variable; the error of the prediction using the model again is called permuted prediction error (PPE); in the third step, the average values of the obtained J NPE and PPE are calculated, denoted as MNPE and MPPE, when MNPE>MPPE, the permutation variable is removed as a non-information or interference variable, and when MNPE≤MPPE, the permutation variable is retained. For the retained variables (j spectral variables), the Mann-Whitney U method is used to test whether the distribution between the J NPE and PPE is significant, and the p value of the retained variable is obtained. The conditional synergetic score (COSS) of the retained variable is evaluated, that is, the logarithm of the negative p value (COSS=-lgp). The greater the COSS value, the more important the retained variable. For the retained variables with a COSS value less than 1, they are directly discarded. For the retained variables with a COSS value greater than 1 and less than 1.5, they are retained and returned to the mirror permutation step, and another permutation variable is added. The third step is executed until the COSS values of all retained variables are greater than or equal to 1.5. Finally, all retained variables are arranged in descending order according to the COSS values, and the several retained variables with the largest COSS values are the damage characteristic wavelengths.

[0053] As Figure 1As shown in the clustering random frog algorithm, set c, ε and k value, analyze the average reflectance spectrum of fruits, and select several damage characteristic wavelengths. First, randomly select K average reflectance spectra as initial centroids, calculate the distance between each sample (fruit) average reflectance spectrum and each initial centroid, and assign each sample to the group where the nearest initial centroid is located; second, calculate the centroid of each group (the centroid is the average of the average reflectance of all samples in the group), then reassign each sample's average reflectance spectrum to the group where the nearest centroid is located, and calculate the centroid of the current group again and reassign; iterate the second step until the iteration reaches the preset value c. Then, the algorithm randomly selects an initial subset V0 containing Q variables as a starting point. In each iteration, the algorithm randomly generates a number Q* from a normal distribution Norm(Q, hQ), representing the number of variables in the candidate variable subset V*. Then, according to the clustering results, a cross-group selection strategy is used to generate the candidate variable subset V*. Next, the algorithm updates the initial subset V0 according to the relationship between Q and Q*: if Q is equal to Q*, use partial least squares discriminant analysis method (PLS-DLA) to calculate and compare the classification accuracy of V* and V0, if the classification accuracy of V* is greater than V0, replace V* with V0, if the classification accuracy of V* is less than or equal to V0, do not replace; if Q is less than Q*, delete Q*-Q variables related to the minimum regression coefficient from V0; if Q is greater than Q*, select Q-Q* variables related to the maximum regression coefficient from V* and add them to V0. Then, the algorithm generates a random number between 0 and 1, if the random number is greater than or equal to the preset threshold, accept V* as the new feature subset V N+1 ; otherwise, reject V* and keep the variable subset of this iteration. The algorithm also calculates the change in classification accuracy between adjacent iterations, if the change is less than the preset value ε and the iteration number N is greater than the preset value k, stop iteration, otherwise, still iterate until the iteration number N reaches the preset maximum iteration number N Max . Finally, arrange the selection probability SR values of all variables in descending order, and the top several variables with the largest SR values are the damage characteristic wavelengths.

[0054] The two groups of damage characteristic wavelengths selected by the sub-window mirror permutation analysis algorithm and the clustering random frog algorithm are merged into N fruit damage characteristic wavelength images using the merge() function in the OpenCV library. The images are divided into training and validation sets in a ratio of 7:3, and the ratio of damaged fruit characteristic wavelength images to healthy fruit characteristic wavelength images in the validation set is 1:1.

[0055] YOLOv8 is a deep learning-based target detection algorithm with the advantages of fast speed, lightweight, high accuracy, open-source program, etc., and is widely used in industrial detection field. The present application uses a YOLOv8 model to identify the target of fruits with slight damage, which is built into a detection terminal.

[0056] When training the YOLOv8 model with the training set, first, a suitable virtual environment needs to be configured and started to run, then the fruit characteristic wavelength image is imported, Labelimg is used as an image data labeling tool, and a target rectangular frame is drawn at the fruit damage site, and the class label is input, and the labeling is completed. During the labeling process, it is necessary to ensure that the rectangular frame is tightly fitted around the target, because a more realistic labeled frame can help the model training to achieve better expected performance. After labeling is completed, an annotation file will be generated for each image data, and each line of the annotation file represents a target object in the image. The first column of data in each line represents the class index of the target, and the subsequent columns represent the position information of the rectangular frame. Place the labeled image data in the images folder and the corresponding annotation file in the labels folder.

[0057] Before model training, the environment needs to be configured and the parameters need to be adjusted, including model parameters and hyperparameters. Model parameters include the number of convolution kernels and the number of pooling layers, which are fixed, but can be optimized for the target task later. Hyperparameters include learning rate lr, epoch, batch-size, and loss function, etc. Among them, the size of batch-size should be matched with the maximum GPU, which is set to a1 here; epoch should be set according to the convergence of the loss function and mAP accuracy, which is set to a2 here; the learning rate is set to a3.

[0058] When using the validation set for verification, the scores of the F1 models trained by the SMPAA dataset (a set of feature wavelength corresponding images of the sub-window mirror image replacement analysis algorithm) and the CRF dataset (a set of feature wavelength corresponding images of the clustering random frog algorithm) are A1 and A2, respectively.

[0059] Embodiment 2

[0060] The present embodiment proposes a multi-spectral imaging online identification system for slight damage on the surface of fruits, which is used for real-time, rapid, non-destructive, accurate detection and classification. As shown in Figure 3 The system includes a fruit holder and a track 101 for batch transportation of fruits to be tested, and a conveying mechanism 102.

[0061] The fruit bowl and track 101 comprises a fruit bowl and a track, the fruit bowl is located on the track and can move horizontally along the track; the fruit bowl comprises a transparent plastic cup body and a cup bottom containing iron products, the transparent cup body helps the transmission of light and does not cause the model recognition accuracy to be reduced due to shielding the light source; the iron cup bottom is the key for the classification mechanism to operate classification; the sliding track can make the fruit bowl move in the specified direction and will not fall off; the sliding track adopts a lightweight and high-strength plastic material.

[0062] The conveying mechanism 102 comprises a support one 112, a support three 113 and a conveying chain set, the conveying chain set comprises a conveying belt and a motor, and is used for continuously conveying the fruit bowl and track 101 loaded with the fruit to be tested; the rotating shafts at both ends of the conveying belt are fixed on the support one 112 and the support three 113 respectively, one of the rotating shafts is driven by the motor, and the working of the motor is controlled by the control unit; a pair of baffles is installed between the support one 112 and the support three 113. The conveying belt is made of wear-resistant polyurethane (PU) material, the track is fixed with the conveying belt, and the tracks are arranged at equal intervals.

[0063] The sorting mechanism 105, the detection mechanism 104, the classification mechanism 105 and the collection mechanism 106 are sequentially arranged along the movement direction of the conveying belt. The sorting mechanism 103 and the classification mechanism 105 are arranged on the baffle, the detection mechanism 104 is arranged on the support two 114, the support two 114 is located at the middle position of the conveying mechanism 102, and the collection mechanism 106 is arranged on the support three 113.

[0064] The sorting mechanism 103 comprises guide panels 107 Figure 4 on both sides of the conveying belt, the guide panels 107 are rubber rollers in a funnel shape, and are used for guiding the fruit to be tested transported on the conveying mechanism 102 to the center line position of the conveying belt to pass through the detection station.

[0065] The detection mechanism 104 is arranged at the detection station, is used for acquiring a spectral image of the fruit to be tested in real time at the detection station, and transmits the spectral image to a detection terminal, the detection terminal is built-in in the control unit; the detection terminal detects whether the fruit to be tested has slight damage according to spectral data, and transmits a result to the classification mechanism 105. When the fruit to be tested passes through the detection station, the proximity sensor A 108 detects when the fruit enters and leaves the detection area and position information of the fruit in the detection area, and transmits the information to the control unit. The control unit controls the active light source 111 to work first, and controls the running speed and tension of the conveying belt according to the signal of the proximity sensor A 108, so as to ensure that each fruit can be captured by the multispectral camera 109 in the best state. Through such a design, the conveying mechanism 102 of the embodiment can closely cooperate with the multispectral camera 109, realize online identification and classification of slight damage on the surface of the fruit, and improve the efficiency and accuracy of fruit quality detection. See Figure 5 .

[0066] Optionally, the multispectral camera 109 is a multi-lens multispectral camera customized for damage feature wavelengths according to Embodiment 1, different filters are embedded in the multiple lenses to obtain spectral images of different wavebands, each lens is equipped with a CMOS image sensor with 300,000 pixels, and the integration type is snapshot to support fast and accurate collection of image information of fruits during actual detection; the multiple lenses simultaneously shoot the fruits to be detected, and finally the images are transmitted to the detection terminal via a USB3.0 interface; the multispectral camera 109 is built-in with a TEC semiconductor cooler to ensure that the multispectral camera 109 is at a normal working temperature.

[0067] The classification mechanism 105 classifies the fruits to be detected according to whether there is slight damage, and the classified fruits slide into the tracks of the collection mechanism 106. The classification mechanism 105 includes an electromagnet array system 110, and a proximity sensor B 115 is arranged on the baffle at the rear side of the electromagnet array system 110; as shown in Figure 6 The classification mechanism 105 includes an electromagnet array system 110, and a proximity sensor B 115 is arranged on the baffle at the rear side of the electromagnet array system 110; as shown in

[0068] The collection mechanism 106 is specifically two slides.

[0069] Embodiment 3

[0070] This example takes the industrial pipeline classification of slightly damaged and undamaged strawberries as an example, and adopts the multispectral imaging online identification system for slight damage on the surface of fruits according to the present application to realize the online rapid detection and classification of slightly damaged strawberries, which is specifically as follows:

[0071] The hyperspectral image data of strawberries was collected using a hyperspectral imaging system (400-1000 nm). First, the prepared healthy strawberries were sequentially collected in the order of the number, and the spectral image data was collected twice for each strawberry. After the image collection, the black and white correction was performed, and the average reflectance spectrum of each healthy area of the strawberry was extracted. Next, the above-mentioned healthy strawberries were artificially damaged, and the same hyperspectral image collection, black and white correction, and average reflectance spectrum extraction of the damaged area of each strawberry were performed twice. The average reflectance spectrum data was used to screen the damage characteristic wavelength of the strawberry.

[0072] After the completion of the strawberry hyperspectral image data collection, the dark reference and white reference were obtained, and the black and white correction of the strawberry hyperspectral image data was performed. Two methods were used to analyze the average reflectance spectrum of the healthy and damaged strawberries, and the damage characteristic wavelength of the strawberry was screened.

[0073] The sub-window mirror replacement analysis algorithm analyzes the average reflectance spectrum of the strawberry, and three damage characteristic wavelengths are screened in the near-infrared band. In the 900-1100 nm band range, the near-infrared spectral characteristics are closely related to water. The cell wall and cell membrane of the fruit damage part are damaged, resulting in changes in water. The spectral bands selected by the clustering random frog algorithm at 737.1 nm and 767.6 nm are related to the third overtone of the OH group, and 963.1 nm is related to water. They are also closely related to water, so the state of the surface water change of the damaged part of the strawberry can be obtained.

[0074] The two groups of damage characteristic wavelengths 958.6 nm, 974 nm, 1025.6 nm and 737.1 nm, 767.6 nm, and 963.1 nm selected by the sub-window mirror replacement analysis algorithm and the clustering random frog algorithm are used to perform image fusion on each group of damage characteristic wavelength images using the merge() function in the OpenCV library. This embodiment generates 200 strawberry damage images, which are divided into a training set and a validation set according to a division ratio of 0.7 and 0.3, and the ratio of the damage fruit characteristic wavelength image to the healthy fruit characteristic wavelength image in the validation set is 1:1.

[0075] The training hardware of the YOLOv8 model uses a Windows 11 version operating system, an internal GPU card GTX3060tap with a memory size of 6G, a CPU core of AMD R7 5800H, and a memory of 32G. To run the supporting files required for YOLOv8, a virtual environment is configured through anaconda, with a Python version of 3.8.18, a PyTorch version of 2.1.0, and a CUDA version of 11.8.0. Before model training, the batch-size size is set to 8, the epoch is set to 1000, and the learning rate lr is set to 0.001.

[0076] When using an independent validation set for verification, the F1 scores of the models trained by the SMPAA and CRF datasets are 0.9583 and 0.930, respectively.

[0077] According to the preferred results, the wavelengths of 958.6 nm, 974 nm, and 1025.6 nm selected by the sub-window mirror image permutation analysis algorithm are selected as the shooting wavelengths of the multi-spectral camera, and a three-lens multi-spectral camera is customized according to the three wavelengths.

[0078] After the to-be-tested strawberries are placed in the fruit holders in the horizontal direction behind the sorting mechanism 103, the fruit holders on the slide are arranged into a column along the center line of the conveying belt by the sorting mechanism 103. When the to-be-tested fruits pass through the detection station, the proximity sensor A108 in the detection mechanism 104 detects when the fruit holder enters and leaves the detection area, as well as the position information of the fruit holder in the detection area. When the fruit holder passes directly below the multi-spectral camera 109, the proximity sensor A108 sends a signal to the control unit, which first controls the active light source 111 to work, and then controls the running speed and tension of the conveying belt according to the signal of the proximity sensor A108, to ensure that each strawberry is captured by the multi-spectral camera 109 in the best state. The spectral image is transmitted to the detection terminal, and the YOLOv8 model performs damage image recognition (if the i-th fruit holder is identified as having slight damage, the fruit holder is recorded as the i+m-th fruit holder when it reaches the area opposite the two electromagnet array systems 110), and then the detection result is transmitted to the control unit, which controls the classification mechanism 105 to act according to the inspection result, such as Figure 7The classification mechanism 105 is shown in FIG. 2. The proximity sensor B 115 is responsible for monitoring whether the strawberry has passed the opposite area of the electromagnet array system 110, and sends a signal to the control unit. The healthy strawberry and the slightly damaged strawberry are respectively captured by the corresponding electromagnet array system 110 according to the signal of the control unit. In the initial stage, the electromagnet generates a strong magnetic field force by applying a large current to ensure the effective capture of the fruit basket in front of it. As the fruit basket gradually approaches the electromagnet, the current in the electromagnet coil is gradually reduced in a proportional manner. Finally, when the fruit basket approaches the preset position, the current is reduced to zero, achieving a smooth transition. Then, the fruit basket and the classified strawberry carried by the conveyor belt continue to move forward. When the fruit basket reaches a certain inclination angle (i.e., at this time the fruit basket is opposite to the entrance of the chute), the strawberry naturally slides out of the fruit basket under the action of gravity, and is finally collected by the special collection panel of the collection mechanism 106.

[0079] The classification results of the embodiment are shown in Table 1:

[0080] Table 1 Classification results of the embodiment

[0081] Classification Total number of samples Number of correct classifications Number of incorrect classifications Number of accurate classifications Healthy strawberries 500 488 12 97.6％ Mildly damaged strawberries 500 487 13 97.4％ Overall 1000 975 25 97.5％

[0082] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present application shall be included in the protection scope of the present application.

[0083] The above describes the specific embodiments of the present application in conjunction with the accompanying drawings, but is not intended to limit the protection scope of the present application. Those skilled in the art should understand that various modifications or changes made by those skilled in the art on the basis of the technical solutions of the present application without creative labor are still within the protection scope of the present application.

Claims

1. A multi-spectral imaging online identification system for slight damage on the surface of fruits, characterized in that, The application relates to a fruit sorting system. The fruit sorting system comprises a conveying mechanism (102), a material arranging mechanism (103), a detecting mechanism (104), a sorting mechanism (105) and a collecting mechanism (106). The detecting mechanism (104) comprises a proximity sensor A (108), a multi-spectrum camera (109) and an active light source (111), wherein the information of the fruit to be detected detected by the proximity sensor A (108) is transmitted to a control unit, the control unit controls the operation of the active light source (111), and the running speed and tension of the conveying belt are controlled according to the signal of the proximity sensor A (108), so that each fruit to be detected can be captured by the multi-spectrum camera (109) in the best state; the multi-spectrum camera (109) is a multi-lens multi-spectrum camera customized according to the screening damage characteristic wavelength, different filters are embedded in the multiple lenses, and each lens is equipped with a CMOS image sensor; the multi-spectrum camera (109) obtains the spectral image of the fruit to be detected and sends the spectral image to a detection terminal, and a YOLOv8 model in the detection terminal performs damage image recognition; The screening damage characteristic wavelength is as follows: the hyperspectral image data of healthy fruits is collected in sequence according to the number order, the collected images are subjected to black and white correction after two times of hyperspectral image collection of each fruit, and the average reflectivity spectrum of the healthy area of each fruit is extracted; then, the above-mentioned healthy fruits are slightly damaged artificially, and the same two times of hyperspectral image collection, black and white correction and extraction of the average reflectivity spectrum of the damage area of each fruit are carried out; and the average reflectivity spectrum data is used for screening the fruit damage characteristic wavelength; The screening fruit damage characteristic wavelength is as follows: two types of algorithms, namely a sub-window mirror image replacement analysis algorithm and a clustering random frog algorithm, are used to analyze the average reflectivity spectrum of the healthy fruits and the damaged fruits, the wavelength spectrum with the top three COSS values obtained by the sub-window mirror image replacement analysis algorithm and the wavelength spectrum with the top three SR values obtained by the clustering random frog algorithm are respectively used as the damage characteristic wavelengths screened by the two types of algorithms. ​ The sub-window mirror image permutation analysis algorithm obtains the spectral bands with top three COSS values, specifically: setting the number of spectral variables j and the number of repetitions N; assuming that the sub-data set obtained by sampling the average reflectivity spectrum of the fruit in the first step Monte Carlo contains all spectral sub-data sets with the number of spectral variables j, a total of J; in the second step of establishing the PLS-LDA classification model, there are J PLS-LDA classification models containing the number of spectral variables j, at this time, the prediction error of the prediction set of the J models is called normal prediction error NPE, in addition, the spectrum matrix of the prediction set of the J models is subjected to mirror image permutation, a variable in the j spectral variables is selected as a permutation variable, and the prediction error of the model is used again; in the third step, the average values of J NPE and PPE are calculated and recorded as MNPE and MPPE, when MNPE>MPPE, the permutation variable is removed as an informationless or interference variable, when MNPE≤MPPE, the permutation variable is retained, and whether the distribution between J NPE and PPE is significant is tested, the p value of the retained variable is obtained, the condition cooperative score COSS of the retained variable is evaluated, the retained variable with a COSS value less than 1 is directly discarded, the retained variable with a COSS value greater than 1 and less than 1.5 is retained and subjected to mirror image permutation, and another permutation variable is added, and the third step is continued to be executed until the COSS values of all retained variables are greater than or equal to 1.5; finally, all retained variables are arranged in descending order according to the COSS values, and the retained variables with top three COSS values are damage characteristic wavelengths; The mirror image permutation is specifically: under a certain wave band, for the average reflectivity of a single sample, if it is greater than the average reflectivity of the whole, it is subtracted by twice the difference between the value and the average reflectivity of the whole, if it is less than the average reflectivity of the whole, it is added by twice the difference between the value and the average reflectivity of the whole, and if it is exactly equal to the average reflectivity of the whole, it remains unchanged.

2. The online multi-spectral imaging system for slight damage identification on fruit surface according to claim 1, characterized in that, The rotating shafts at both ends of the conveying belt are fixed on the support one (112) and the support three (113), one of the rotating shafts is driven by a motor, and the operation of the motor is controlled by a control unit; a pair of baffles is installed between the support one (112) and the support three (113), the sorting mechanism (103) and the classification mechanism (105) are arranged on the baffles, the detection mechanism (104) is arranged on the support two (114), the support two (114) is located at the middle position of the conveying mechanism (102), and the collecting mechanism (106) is arranged on the support three (113).

3. The online multi-spectral imaging system for the detection of superficial defects on fruit surfaces according to claim 1, characterized in that, The clustering random frog algorithm obtains the spectral bands with top three SR values, specifically: Setting c, ε and k values; the first step, randomly select K average reflectivity spectrum as the initial centroid, calculate the average reflectivity spectrum of each sample to each initial centroid distance, each sample is assigned to the nearest initial centroid group, the second step, calculate the centroid of each group, then the average reflectivity spectrum of each sample is re-assigned, assigned to the nearest centroid of the sample average reflectivity spectrum distance group, again calculate the centroid of the current group and re-assignment, the second step is iterated until the iteration number reaches the preset value c value; then, the algorithm randomly selects an initial subset V0 containing Q variables as the starting point, in each iteration, the algorithm randomly generates a number Q* from the normal distribution Norm(Q, hQ), representing the number of variables in the candidate variable subset V*; then, update the initial subset V0 according to the relationship between Q and Q*; then, generate a random number between 0 and 1, if the random number is greater than or equal to the preset threshold, accept V* as the new feature subset V N+1 , otherwise, reject V* and keep the variable subset of this iteration; the algorithm also calculates the change of classification accuracy of adjacent two iterations, if the change is less than the preset value ε and the iteration number is greater than the preset value k, stop iteration, otherwise, still iteration, until the iteration number reaches the preset maximum iteration number; after iteration, arrange the size of the selection probability SR value of all variables in descending order, the top three variables of SR value are damage characteristic wavelength.

4. The online multi-spectral imaging system for slight damage identification on fruit surface according to claim 3, characterized in that, The initial subset V0 is updated according to the relationship between Q and Q*, specifically: If Q is equal to Q*, the classification accuracies of V* and V0 are calculated and compared, if the classification accuracy of V* is greater than V0, V* replaces V0, if the classification accuracy of V* is less than or equal to V0, no replacement is performed; If Q is less than Q*, Q*-Q variables associated with the minimum regression coefficient are deleted from V0; If Q is greater than Q*, Q-Q* variables associated with the maximum regression coefficient in V* are selected and added to V0.

5. The online multi-spectral imaging system for slight damage identification on fruit surface according to claim 1, characterized in that, Through the sub-window mirror permutation analysis algorithm and the clustering random jumping frog algorithm, two groups of damage characteristic wavelengths are screened out respectively, the images of each group of damage characteristic wavelengths are fused, N characteristic wavelength images of fruit damage are obtained, and the training set and the verification set of the YOLOv8 model are divided according to the ratio of 7:3, and the ratio of the characteristic wavelength images of the damaged fruits to the characteristic wavelength images of the healthy fruits in the verification set is 1:

1.

6. The online multi-spectral imaging system for slight damage identification on fruit surface according to claim 1, characterized in that, The fruit tray and track (101) comprises a fruit tray and a track, the fruit tray is located on the track and can move horizontally along the track.

7. The online multi-spectral imaging system for the detection of superficial defects on fruit surfaces according to claim 1, characterized in that, The material sorting mechanism (103) comprises guide panels (107) arranged on both sides of the conveying belt, which are used to guide the transported fruits to be tested to the center line position of the conveying belt.

8. The online multi-spectral imaging system for slight damage identification on fruit surface according to claim 1, characterized in that, The classification mechanism (105) comprises a set of electromagnet array systems (110) arranged on both sides of the conveying belt, and a proximity sensor B (115) is arranged on the rear side of the electromagnet array system (110); the electromagnet array system (110) is integrated with a controller, which can receive the instructions of the control unit and apply precise magnetic field force to the fruit tray to move the slightly damaged fruits and the intact fruits to different sides.

9. The online multi-spectral imaging system for slight damage identification on fruit surface according to claim 8, characterized in that, The working principle of the electromagnet array system (110) is as follows: in the initial stage, the electromagnet rapidly establishes sufficient magnetic field force by passing a large current, so as to ensure that the fruit tray is stably attracted and moved; Subsequently, the current is gradually reduced in a proportional manner, and the acceleration of the fruit tray is slowed down.

10. The online multi-spectral imaging system for slight damage identification on fruit surface according to claim 1, characterized in that, The collecting mechanism (106) is specifically two slides.

Images