Litchi lossless sorting method and device, storage medium and computer equipment
Through multi-spectral imaging and deep learning, the lychee quality prediction model is solved, and the problems of poor timeliness, high cost and strong subjectivity of artificial detection of lychee quality are achieved, and the quality of lychee is quickly and accurately evaluated, which improves production efficiency and sorting accuracy. .
Patent Information
- Application Number
- CN202411133719.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-19
- Publication Date
- 2024-11-29
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
In the prior art, lychee quality inspection relies on artificial naked eyes, which has poor timeliness, high cost and strong subjectivity, and cannot meet the needs of modern agriculture.
Using multi-spectral imaging and deep learning methods, the visible, infrared and ultraviolet images of lychees are obtained, and the effective image area is extracted through the image segmentation network, the target image features are extracted, and input to the trained prediction model to output the actual level of lychees. .
It has achieved rapid, accurate and non-invasive quality prediction of litchi quality, improved the efficiency and sorting accuracy of the production line, significantly improved the efficiency and accuracy of litchi quality evaluation, and promoted the development of the litchi intelligent sorting industry.
Smart Images

Figure CN119049041A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence technology, in particular to the field of nondestructive testing technology, and in particular to a method, device, computer equipment and storage medium for nondestructive sorting of litchi. Background Art
[0002] Lychee is one of the native fruits of China, with a long history and rich cultural connotations. Lychee cultivation and related industries are the key industries developed by Guangdong Province, and lychee production occupies an important position in the economic development of Guangdong Province. In recent years, the output of lychees has remained at a high level, but with the continuous improvement of people's living standards, the market demand for high-quality lychees is also growing.
[0003] However, if the quality of litchi cannot be accurately distinguished, the post-harvest added value of litchi will be directly affected. The traditional method is to identify the quality of litchi by manual visual inspection. This manual inspection method has defects such as poor timeliness, high cost, and strong subjectivity, and can no longer meet the needs of modern agricultural development. In order to adapt to market demand and promote the modernization of the litchi industry, an efficient and accurate litchi quality assessment method is urgently needed. Summary of the invention
[0004] Based on this, the main purpose of this application is to provide a method, device, computer equipment and storage medium for non-destructive sorting of litchi based on multispectral imaging and deep learning, so as to solve the problems of poor timeliness, high cost and strong subjectivity in manual detection of litchi quality in the prior art.
[0005] To achieve the above-mentioned objectives, the first aspect of the present application provides a lossless sorting method for litchi, comprising: obtaining a multispectral image of litchi, wherein the multispectral image includes at least two of a visible light image, an infrared light image and an ultraviolet light image; inputting the multispectral image into an image segmentation network for segmentation processing to obtain a valid image area of the litchi; extracting target image features of the valid image area; inputting the target image features into a trained prediction model, and outputting the actual grade of the litchi through the prediction model.
[0006] Optionally, extracting target image features of the effective image area includes: extracting initial image features of the effective image area; and filtering out target image features from the initial image features, wherein the number of the initial image features is greater than the number of the target image features.
[0007] Optionally, the inputting the multispectral image into an image segmentation network for segmentation processing to obtain the effective image area of the litchi includes: converting the multispectral image into a grayscale image; binarizing the grayscale image to obtain a binary image; performing morphological operations on the binary image to obtain a morphological image, wherein the morphological operations include at least one of dilation, erosion, opening and closing operations; and extracting the edges of the morphological image by an edge detection algorithm to obtain the effective image area of the litchi.
[0008] Optionally, extracting the target image features of the effective image area includes: performing RGB channel separation on the effective image area, extracting RGB image data of red, green and blue channels respectively, and performing statistical calculation on the RGB image data of each channel to extract color features of the effective image area as target image features, the color features include at least one of the number of colors and their distribution ratio, color dispersion, color uniformity, brightness, average color and main color; and / or, calculating the gray level co-occurrence matrix of the effective image area, and extracting texture features of the effective image area as target image features, the texture features include at least one of contrast, correlation, entropy and uniformity; and / or, performing contour analysis on the effective image area to obtain geometric features of the effective image area as target image features, the geometric features include at least one of shape, size and surface area; and / or, performing spectral analysis on the effective image area to obtain spectral features of the effective image area as target image features, the spectral features include at least one of reflectivity, refractive index and absorptivity.
[0009] Optionally, before inputting the target image features into the trained prediction model and outputting the actual grade of the lychees through the prediction model, it also includes: pre-identifying and marking the grade of the lychees as a marked grade; inputting the target image features into the initial prediction model; analyzing the contribution of the target image features to the prediction results based on the marked grade; removing features with low contribution or negative impact on the prediction results, and retaining features with high contribution to obtain a trained prediction model.
[0010] Optionally, the step of inputting the target image features into a trained prediction model and outputting the actual grade of the lychees through the prediction model comprises: inputting the target image features into a trained prediction model to obtain the predicted grade output by each decision tree in the prediction model; and screening out the predicted grades with the largest number from the predicted grades as the actual grade of the lychees.
[0011] Optionally, after inputting the target image features into a trained prediction model and outputting the actual grade of the lychees through the prediction model, the method further includes: providing real-time feedback of the actual grade of the lychees.
[0012] The second aspect of the present application provides a lossless lychee sorting device, comprising: an image acquisition module, used to acquire a multispectral image of lychees, wherein the multispectral image includes at least two of a visible light image, an infrared light image and an ultraviolet light image; an image processing module, used to input the multispectral image into an image segmentation network for segmentation processing to obtain a valid image area of the lychees; a feature extraction module, used to extract target image features of the valid image area; and a grade determination module, used to input the target image features into a trained prediction model, and output the actual grade of the lychees through the prediction model.
[0013] Optionally, the feature extraction module is specifically used to: extract initial image features of the effective image area; and filter out target image features from the initial image features, wherein the number of the initial image features is greater than the number of the target image features.
[0014] Optionally, the image processing module is specifically used to: convert the multispectral image into a grayscale image; binarize the grayscale image to obtain a binary image; perform morphological operations on the binary image to obtain a morphological image, wherein the morphological operations include at least one of dilation, erosion, opening and closing operations; extract the edges of the morphological image through an edge detection algorithm to obtain the effective image area of the litchi.
[0015] Optionally, the feature extraction module is specifically used to: perform RGB channel separation on the effective image area, extract RGB image data of red, green and blue channels respectively, and perform statistical calculations on the RGB image data of each channel to extract color features of the effective image area as target image features, wherein the color features include at least one of the number of colors and their distribution ratio, the degree of color dispersion, color uniformity, brightness, average color and main color; and / or, calculate the grayscale co-occurrence matrix of the effective image area, and extract texture features of the effective image area as target image features, wherein the texture features include at least one of contrast, correlation, entropy and uniformity; and / or, perform contour analysis on the effective image area to obtain geometric features of the effective image area as target image features, wherein the geometric features include at least one of shape, size and surface area; and / or, perform spectral analysis on the effective image area to obtain spectral features of the effective image area as target image features, wherein the spectral features include at least one of reflectivity, refractive index and absorptivity.
[0016] Optionally, the lychee lossless sorting device also includes: a model training module, used to pre-identify and mark the grade of the lychee as a marked grade; input the target image features into an initial prediction model; according to the marked grade, analyze the contribution of the target image features to the prediction results; remove features with low contribution or negative impact on the prediction results, and retain features with high contribution to obtain a trained prediction model.
[0017] Optionally, the grade determination module is specifically used to: input the target image features into a trained prediction model to obtain the predicted grades output by each decision tree in the prediction model; and screen out the predicted grades with the largest number from the predicted grades as the actual grades of the litchi.
[0018] Optionally, the lychee lossless sorting device further comprises a feedback module for providing real-time feedback of the actual grade of the lychees.
[0019] The third aspect of the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the lychee lossless sorting method as described in any one of the above items are implemented.
[0020] The fourth aspect of the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the method for lossless lychee sorting as described in any one of the above items are implemented.
[0021] The non-destructive sorting method, device, computer equipment and computer-readable storage medium provided by the present application have the following beneficial effects: by using a lychee quality prediction model based on multispectral imaging and deep learning, the grading of lychee quality can be realized, which can solve the problems of poor timeliness, high cost and strong subjectivity in the manual detection of lychee quality in the prior art. Through the non-destructive sorting method of lychee based on artificial intelligence, it is possible to achieve rapid, non-invasive and accurate quality prediction of lychee, thereby improving the efficiency and sorting accuracy of the production line. In addition, by obtaining multi-spectral images of lychee, such as at least two of visible light images, infrared images and ultraviolet images, multiple groups of spectral images can be obtained, so as to provide data for subsequent prediction analysis from multiple angles and in a coordinated manner, so that the lychee grade results output by the prediction model are more accurate and reliable, thereby significantly improving the efficiency and accuracy of lychee quality assessment and promoting the high-quality development of the lychee intelligent sorting industry. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] Figure 1 is a schematic diagram of a non-destructive sorting method for litchi in one embodiment of the present application;
[0023] Figure 2 This is a structural block diagram of a non-destructive litchi sorting device in one embodiment of the present application;
[0024] Figure 3 It is a schematic block diagram of the structure of a computer device according to an embodiment of the present application.
[0025] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0026] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0027] Reference Figure 1 The embodiment of the present application provides a lossless lychee sorting method, comprising: step S101, obtaining a multispectral image of lychees; step S102, inputting the multispectral image into an image segmentation network for segmentation processing to obtain a valid image area of lychees; step S103, extracting target image features of the valid image area; step S104, inputting the target image features into a trained prediction model, and outputting the actual grade of the lychees through the prediction model.
[0028] The non-destructive sorting method for litchi provided in the embodiment of the present application realizes the grade sorting of litchi quality through the litchi quality prediction model based on multi-spectral imaging and deep learning, which can solve the problems of poor timeliness, high cost, and strong subjectivity in the manual detection of litchi quality in the prior art. Through this non-destructive sorting method for litchi based on artificial intelligence, it is possible to achieve rapid, non-invasive, and accurate quality prediction of litchi, thereby improving the efficiency and sorting accuracy of the production line. In addition, by obtaining multi-spectral images of litchi, such as at least two of visible light images, infrared images, and ultraviolet images, multiple groups of spectral images can be obtained to provide data for subsequent prediction analysis from multiple angles and in a coordinated manner, so that the litchi grade results output by the prediction model are more accurate and reliable, thereby significantly improving the efficiency and accuracy of litchi quality assessment and promoting the high-quality development of the litchi intelligent sorting industry.
[0029] In step S101, a multispectral image of litchi is acquired.
[0030] In some embodiments, the multispectral image includes at least one of a visible light image, an infrared light image, and an ultraviolet light image. In other embodiments, the multispectral image includes at least two of a visible light image, an infrared light image, and an ultraviolet light image. In yet other embodiments, the multispectral image includes a visible light image, an infrared light image, and an ultraviolet light image. Among them, the visible light image refers to an image obtained by irradiating litchi with a visible light source (e.g., an RGB light source, i.e., a red, green, and blue light source) and photographing with a camera having a visible light imaging system; the infrared light image refers to an image obtained by irradiating litchi with an infrared light source (e.g., a near-infrared light source and / or a far-infrared light source) and photographing with a camera having an infrared imaging system; the ultraviolet light image refers to an image obtained by irradiating litchi with an ultraviolet light source (e.g., a near-ultraviolet light source and / or a far-ultraviolet light source) and photographing with a camera having an ultraviolet imaging system. In some embodiments, the camera may have an imaging system with only one type of light source, or may have an imaging system with two or more types of light sources (e.g., a multispectral imaging system having visible light, infrared, and ultraviolet at the same time).
[0031] Among them, the color and texture information of the litchi surface can be captured through the visible light image; the infrared light image can penetrate the litchi skin, capture the internal structure and characteristics of the litchi, and identify possible defects or maturity; ultraviolet light can stimulate the fluorescent reaction of the litchi skin and interior, and the ultraviolet light image can reveal the subtle structure of the litchi skin and possible quality problems inside. Therefore, by obtaining a multispectral image of the litchi, and making the multispectral image include at least two of the visible light image, infrared light image and ultraviolet light image, multiple groups of spectral images can be obtained at the same time, and the surface and internal characteristics of the litchi can be extracted at the same time, thereby providing data for subsequent prediction and analysis from multiple angles and in a coordinated manner, making the litchi grade results output by the prediction model more accurate and reliable, thereby significantly improving the efficiency and accuracy of litchi quality assessment and promoting the high-quality development of the litchi intelligent sorting industry.
[0032] In order to make the multi-spectral image of the litchi obtained in step S101 clearer and more accurate, before step S101, it may include: a plurality of light sources are evenly arranged around, wherein the light sources include at least two of a visible light source, an infrared light source and an ultraviolet light source. In some embodiments, there may be more than two visible light sources, for example, four, which are respectively arranged on the upper, lower, left and right four surfaces of the litchi. In other embodiments, there may be more than two infrared light sources, for example, four, which are respectively arranged on the upper, lower, left and right four surfaces of the litchi. In yet other embodiments, there may be more than two ultraviolet light sources, for example, two, which are respectively arranged on the upper and lower surfaces of the litchi. Among them, the positions of the visible light source, the infrared light source, and the ultraviolet light source may be completely staggered, partially overlapped, or completely overlapped, and may be adjusted according to actual needs to ensure that various light sources can evenly cover the surface of the litchi.
[0033] In addition, before step S101, the method may further include: using a multispectral camera to simultaneously capture at least two of the visible light image, the infrared light image, and the ultraviolet light image. In some embodiments, a multispectral filter may be provided in the multispectral camera. Through the multispectral camera, more than two spectral images can be captured simultaneously, and through the setting of the multispectral filter, it can be ensured that the image of each light source can be captured separately, simplifying the image capture process and making the captured image clearer.
[0034] In step S102, the multispectral image is input into an image segmentation network for segmentation processing to obtain the effective image area of the litchi. In some embodiments, the image segmentation network can be an image segmentation network such as Fun, Unet or Unet++ based on deep learning. In some embodiments, the multispectral image is input into the image segmentation network for segmentation processing to remove invalid areas such as background or other debris, identify and retain the effective area belonging to the litchi part, thereby obtaining the effective image area of the litchi. Thus, through the image segmentation network, images of different spectra can be fused and analyzed, which can improve the accuracy of removing background or other debris, thereby improving the accuracy and recognition efficiency of the effective image area of the litchi.
[0035] In some embodiments, the multispectral image may be a color image, and inputting the multispectral image into an image segmentation network for segmentation processing to obtain the effective image region of litchi may include: converting the multispectral image into a grayscale image; binarizing the grayscale image to obtain a binary image; and extracting the edge of the binary image using an edge detection algorithm to obtain the effective image region of litchi. Thus, by converting the multispectral image into a grayscale image and binarizing it, the obtained binary image can be better used for extracting the effective image region of litchi.
[0036] In some embodiments, the multispectral image may be a color image, and the multispectral image is input into an image segmentation network for segmentation processing to obtain the effective image region of the litchi, which may include: converting the multispectral image into a grayscale image; binarizing the grayscale image to obtain a binarized image; performing morphological operations on the binarized image to obtain a morphological image; and extracting the edge of the morphological image by an edge detection algorithm to obtain the effective image region of the litchi. Thus, by converting the multispectral image into a grayscale image and performing binarization processing, the image can be more effectively morphologically operated, and after the morphological operation, the edge and related detail features of the litchi can be better extracted, thereby obtaining a more accurate effective image region.
[0037] In some embodiments, the morphological operation may include at least one of dilation, erosion, opening and closing operations. Among them, the dilation operation can expand the white area in the image, fill small holes, and connect adjacent areas; the erosion operation can shrink the white area in the image, remove small noise points, and separate adjacent areas; the opening operation is to erode first and then dilate, which is mainly used to remove small noise in the image; the closing operation is to dilate first and then erode, which is mainly used to fill small holes in the image and connect adjacent areas. Therefore, through the above morphological operations, the image before processing can be further improved (for example, filling small holes or removing noise), so that the processed image is clearer and more accurate.
[0038] In some embodiments, the multispectral image can be directly a grayscale image, and the multispectral image is input into the image segmentation network for segmentation processing to obtain the effective image area of litchi, which can include: binarizing the grayscale image to obtain a binarized image. In other embodiments, it can also include: performing morphological operations on the binarized image to obtain a morphological image. In some other embodiments, it can also include: extracting the edge of the binarized image or the morphological image by an edge detection algorithm to obtain the effective image area of litchi. Therefore, since the multispectral image is originally a grayscale image, there is no need to perform a grayscale processing step, and the multispectral image can be directly binarized, thereby extracting the edge of the binarized image to obtain the effective image area of litchi. The binarized image can also be further subjected to morphological operations. After morphological operations, the edges and related detail features of litchi can be better extracted, thereby obtaining a more accurate effective image area.
[0039] In some embodiments, the edge detection algorithm may include a multi-level edge detection algorithm, such as a Canny algorithm, a Sobel algorithm, or a Laplace algorithm, etc. The multi-level edge detection algorithm can further improve the accuracy of extracting the edges and related detail features of the litchi, so that the obtained effective image area is more accurate and reliable.
[0040] In step S103, target image features of the effective image area are extracted.
[0041] In some embodiments, extracting the target image features of the effective image area may include: extracting the initial image features of the effective image area; and screening the target image features from the initial image features. The number of the initial image features may be the same as the number of the target image features. That is, in some cases, if the extracted initial image feature data are all good (for example, all meet the preset condition interval), the initial image features may be directly used as the target image features, or all the initial image features may be selected during screening to serve as the target image features.
[0042] In other embodiments, the number of initial image features may be greater than the number of target image features. In this case, data with poor results in the initial image features (e.g., data that does not meet the preset condition interval) may be excluded, and data with better results (e.g., data that meets the preset condition interval) may be screened out as target image features. Exemplarily, lychee X with grade A has target image features that meet the requirements, namely, abc (where d in the initial image features does not meet the preset conditions), and lychee Y with grade A also has target image features that meet the requirements, namely, acd (where b in the initial image features does not meet the preset conditions), but both have grades A and should be classified into the same quality grade; if the target image feature is fixed to abc, then lychee Y will inevitably be ultimately determined to be a quality grade that does not belong to lychee X; and if the target image feature is fixed to acd, then lychee X will inevitably be ultimately determined to be a quality grade that does not belong to lychee Y. In this case, the final determination result will be inconsistent with the actual situation (lychees X and Y belong to the same quality grade).
[0043] Therefore, by adding the step of screening target image features from the initial image features, better target image features can be adaptively screened out when there are multiple optional features (features that affect the quality of litchi), thereby ensuring the consistency of grading of litchis with different features but the same quality.
[0044] In some embodiments, extracting the target image features of the effective image area may include: performing RGB channel separation on the effective image area, extracting RGB image data of the red, green and blue channels respectively, and performing statistical calculations on the RGB image data of each channel to extract the color features of the effective image area as the target image features. In other embodiments, the color features may include at least one of the number of colors and their distribution ratio, the degree of color dispersion, color uniformity, brightness, average color and main color. Among them, color data is extracted through RGB channels, and statistical parameters such as histogram, mean and standard deviation of each channel are calculated as color features (i.e., target image features). The color and texture information of the lychee surface can be obtained through these color features to reflect the appearance quality of the lychee.
[0045] In some embodiments, extracting the target image features of the effective image area may include: calculating the Gray-Level Co-occurrence Matrix (GLCM) of the effective image area, and extracting the texture features of the effective image area as the target image features. In other embodiments, the texture features may include at least one of contrast, correlation, entropy, and uniformity. By calculating and analyzing the gray-level co-occurrence matrix, the texture features of the litchi can be extracted for evaluating the surface structure and texture of the litchi.
[0046] In some embodiments, extracting the target image features of the effective image area may include: performing contour analysis on the effective image area to obtain geometric features of the effective image area as the target image features. In other embodiments, the geometric features may include at least one of shape, size, and surface area. The surface area of the litchi can be calculated by extracting the contour of the effective image area. Through contour analysis, the geometric features of the litchi can be obtained to facilitate the evaluation of its shape, size, weight, etc.
[0047] In some embodiments, extracting the target image feature of the effective image area may include: performing spectral analysis on the effective image area to obtain the spectral feature of the effective image area as the target image feature. In other embodiments, the spectral feature includes at least one of reflectivity, refractive index and absorptivity. The spectral feature can further reflect the surface morphology of the litchi.
[0048] In some embodiments, the target image features of the effective image area can be extracted by the above-mentioned multiple methods at the same time, so that the target image features can include at least two of the color features, texture features, geometric features and spectral features. Thus, at least two of the color features, texture features, geometric features and spectral features of litchi can be fused together, and the accuracy of the litchi sorting results can be further improved through the multi-layer feature fusion technology.
[0049] In step S104, the target image features are input into the trained prediction model, and the actual grade of the litchi is output through the prediction model. Among them, the prediction model may include a prediction model based on deep learning, such as a random forest model, an XGBoost (distributed gradient boosting library) model, or a LightGBM (lightweight gradient boosting machine) model. Through the prediction model based on deep learning, large-scale data and features can be processed, with higher analysis efficiency and judgment accuracy, thereby improving the efficiency of the litchi sorting process and the accuracy and reliability of the results.
[0050] In some embodiments, there are N prediction models, where N is an odd number greater than or equal to 3. In some embodiments, inputting the target image features into the trained prediction model, and outputting the actual grade of the litchi through the prediction model, may include: inputting the target image features into the trained N prediction models respectively, and each prediction model outputs the predicted grade of the litchi respectively; and screening out the predicted grades with the largest number from the predicted grades as the actual grade of the litchi. Thus, through the collaborative prediction of multiple prediction models, the accuracy of the litchi grade sorting results can be further improved to improve the reliability of the sorting method.
[0051] In some embodiments, inputting the target image features into the trained prediction model, and outputting the actual grade of the litchi through the prediction model may include: inputting the target image features into the trained prediction model to obtain the predicted grades output by each decision tree in the prediction model; and screening out the predicted grades with the largest number from the predicted grades as the actual grade of the litchi. Thus, by setting up multiple decision trees, the predicted grades of the litchi can be obtained from multiple angles, so as to determine the actual grade of the litchi through comparative analysis, so as to improve the accuracy and reliability of the litchi grade sorting.
[0052] In some embodiments, before inputting the target image features into the trained prediction model and outputting the actual grade of litchi through the prediction model, it also includes: pre-identifying and marking the grade of litchi as the marked grade; inputting the target image features into the initial prediction model; analyzing the contribution of the target image features to the prediction results according to the marked grade; removing features with low contribution or negative impact on the prediction results, and retaining features with high contribution to obtain the trained prediction model. Thus, by inputting a large amount of data into the prediction model in advance, the prediction model can determine the correlation between each parameter and the quality of litchi through statistical analysis and machine learning algorithms, thereby screening out the feature parameters that have a significant impact on the quality of litchi, and ensuring that the grading results are accurate and reliable.
[0053] It should be noted that the contribution of the target image feature to the prediction result can be set according to actual needs. For example, it can be set that when the influence of a certain target image feature on the prediction result reaches a preset standard value or above, the contribution of the target image feature to the prediction result is high, when the influence of a certain target image feature on the prediction result is below the preset standard value, the contribution of the target image feature to the prediction result is low, and when a certain target image feature has a reverse effect on the prediction result (i.e., the result obtained according to the data of the target image feature shows that the grade of lychee is A, but the actual grade of the actual lychee (such as the artificially marked grade) is B), the target image feature is considered to have a negative impact on the prediction result. Among them, the preset standard value can be adjusted according to actual needs, for example, it can be 10%, 20%, 30%, 50%, etc.
[0054] In some embodiments, pre-identifying and marking the grade of litchi as the marked grade may include: manually identifying and marking the grade of litchi as the marked grade. Thus, the grade can be manually identified in advance as data for training the prediction model, thereby improving the sorting accuracy of the prediction model.
[0055] In other embodiments, analyzing the contribution of the target image features to the prediction results according to the labeling level may include: analyzing the contribution of the target image features to the prediction results according to the labeling level using SHAP values (SHapley Additive exPlanations). The SHAP value is used to analyze the contribution of each target image feature to the prediction result, and the analysis can be performed from the following perspectives: (1) global feature importance - displaying the global feature importance, showing which features have the greatest overall impact on the model prediction; (2) single prediction explanation - explaining the prediction of a single sample, showing the contribution of each feature to the prediction; (3) feature interaction analysis - analyzing the interaction between features to understand the performance of a feature under different conditions.
[0056] In some embodiments, based on the analysis results, such as the SHAP value analysis results, the prediction model can be optimized as follows: (1) feature selection - remove features with low contribution or negative impact on prediction, retain and focus on analyzing features with high contribution; (2) model adjustment - adjust model parameters and optimize model performance according to the distribution and interaction of important features. Thus, a better trained prediction model is obtained, so that the litchi sorting grade output by the prediction model is more accurate and reliable.
[0057] In some embodiments, after inputting the target image features into the trained prediction model and outputting the actual grade of the litchi through the prediction model, the method may further include: feeding back the actual grade of the litchi in real time. For example, the actual grade of the litchi may be fed back to the transmission device in real time, and the transmission device may transmit the litchi to the corresponding channel according to the actual grade of the litchi, so as to achieve the litchi sorting effect.
[0058] In some embodiments, after inputting the target image features into the trained prediction model and outputting the actual grade of the litchi through the prediction model, it may also include: detecting whether the actual grade of the litchi is accurate. In some embodiments, detecting whether the actual grade of the litchi is accurate may include: detecting whether the actual grade of the litchi is accurate when a preset time or a preset count is reached. For example, after the litchi is sorted for a certain time (e.g., 1 hour) or a certain number (e.g., 100) using the sorting method, one or more sorted litchis are manually identified to determine whether the result of the manual identification is consistent with the predicted actual grade of the litchi, if it is consistent, it is accurate, if it is not consistent, it is inaccurate.
[0059] In some embodiments, when it is detected that the actual grade of the litchi is accurate, the sorting can continue; when it is detected that the actual grade of the litchi is inaccurate, after detecting whether the actual grade of the litchi is accurate, it can also include: inputting the results of manual recognition into the prediction model, and further training and optimizing the prediction model. In this way, the prediction model continuously optimizes itself through deep learning, and continuously adjusts the sorting standards and parameters to improve the intelligent level of sorting, thereby improving the accuracy of the litchi sorting results and the sorting efficiency.
[0060] Reference Figure 2 The embodiment of the present application also provides a lossless lychee sorting device, including: an image acquisition module 201, used to acquire a multispectral image of lychees, wherein the multispectral image includes at least two of a visible light image, an infrared light image, and an ultraviolet light image; an image processing module 202, used to input the multispectral image into an image segmentation network for segmentation processing to obtain a valid image area of lychees; a feature extraction module 203, used to extract target image features of the valid image area; a grade determination module 204, used to input the target image features into a trained prediction model, and output the actual grade of the lychees through the prediction model.
[0061] The lychee non-destructive sorting device provided in the embodiment of the present application realizes the grade sorting of lychee quality through the lychee quality prediction model based on multi-spectral imaging and deep learning, which can solve the problems of poor timeliness, high cost, and strong subjectivity in the manual detection of lychee quality in the prior art. Through the lychee non-destructive sorting method based on artificial intelligence, it is possible to achieve rapid, non-invasive, and accurate quality prediction of lychees, thereby improving the efficiency and sorting accuracy of the production line. In addition, by obtaining multi-spectral images of lychees, such as at least two of visible light images, infrared images, and ultraviolet images, multiple groups of spectral images can be obtained to provide data for subsequent prediction analysis from multiple angles and in a coordinated manner, so that the lychee grade results output by the prediction model are more accurate and reliable, thereby significantly improving the efficiency and accuracy of lychee quality assessment and promoting the high-quality development of the lychee intelligent sorting industry.
[0062] In some embodiments, the feature extraction module 203 can be specifically used to: extract the initial image features of the effective image area; and filter out the target image features from the initial image features. The number of the initial image features can be the same as the number of the target image features. That is, in some cases, if the extracted initial image feature data are all good (for example, they all meet the preset condition interval), the initial image features can be directly used as the target image features, or all the initial image features can be selected during the screening to serve as the target image features.
[0063] In other embodiments, the number of initial image features is greater than the number of target image features. In this case, data with poor results in the initial image features (e.g., data that do not meet the preset condition interval) can be excluded, and data with better results (e.g., data that meet the preset condition interval) can be screened out as target image features. By adding the step of screening target image features from the initial image features, in the case of multiple optional features (features that affect the quality of litchi), more optimal target image features can be adaptively screened out, thereby ensuring the consistency of grading of litchis with different features but the same quality.
[0064] In some embodiments, the image processing module 202 can be specifically used to: convert the multispectral image into a grayscale image; perform binarization processing on the grayscale image to obtain a binarized image; perform morphological operations on the binarized image to obtain a morphological image, wherein the morphological operation includes at least one of dilation, corrosion, opening operation and closing operation; extract the edge of the morphological image through an edge detection algorithm to obtain the effective image area of the litchi. Therefore, by converting the multispectral image into a grayscale image and performing binarization processing, the image can be more effectively morphologically operated, and after the morphological operation, the edge and related detail features of the litchi can be better extracted, thereby obtaining a more accurate effective image area.
[0065] In some embodiments, the feature extraction module 203 can also be specifically used to: perform RGB channel separation on the effective image area, extract RGB image data of the red, green and blue channels respectively, and perform statistical calculations on the RGB image data of each channel to extract color features of the effective image area as target image features, the color features include the number of colors and their distribution ratio, the degree of color dispersion, color uniformity, brightness, average color and at least one of the main colors; and / or, calculate the grayscale co-occurrence matrix of the effective image area, and extract texture features of the effective image area as target image features, the texture features include at least one of contrast, correlation, entropy and uniformity; and / or, perform contour analysis on the effective image area to obtain geometric features of the effective image area as target image features, the geometric features include at least one of shape, size and surface area; and / or, perform spectral analysis on the effective image area to obtain spectral features of the effective image area as target image features, the spectral features include at least one of reflectivity, refractive index and absorptivity.
[0066] In some embodiments, the target image features of the effective image area can be extracted by the above-mentioned multiple methods at the same time, so that the target image features can include at least two of the color features, texture features, geometric features and spectral features. Thus, at least two of the color features, texture features, geometric features and spectral features of litchi can be fused together, and the accuracy of the litchi sorting results can be further improved through the multi-layer feature fusion technology.
[0067] In some embodiments, the lychee non-destructive sorting device may further include: a model training module for pre-identifying and marking the grade of lychees as a marking grade; inputting the target image features into the initial prediction model; analyzing the contribution of the target image features to the prediction results according to the marking grade; removing features with low contribution or negative impact on the prediction results, and retaining features with high contribution to obtain a trained prediction model. Thus, by inputting a large amount of data into the prediction model in advance, the prediction model can determine the correlation between each parameter and the quality of lychees through statistical analysis and machine learning algorithms, thereby screening out the characteristic parameters that have a significant impact on the quality of lychees, and ensuring that the grading results are accurate and reliable.
[0068] In some embodiments, the grade determination module 204 can be specifically used to: input the target image features into the trained prediction model to obtain the predicted grades output by each decision tree in the prediction model; and select the predicted grades with the largest number from the predicted grades as the actual grades of the litchi. Thus, by setting up multiple decision trees, the predicted grades of the litchi can be obtained from multiple angles, and the actual grades of the litchi can be determined by comparative analysis to improve the accuracy and reliability of litchi grade sorting.
[0069] In some embodiments, the lychee lossless sorting device may further include a feedback module: used to provide real-time feedback on the actual grade of the lychees. For example, the actual grade of the lychees may be fed back to the transmission device in real time, and the transmission device may transmit the lychees to the corresponding channel according to the actual grade of the lychees, so as to achieve the lychee sorting effect.
[0070] For the specific implementation of each module in the above device embodiment, please refer to the above method embodiment, which will not be repeated here.
[0071] Reference Figure 3 In an embodiment of the present application, a computer device is also provided. The computer device may be a server, and its internal structure may be as follows: Figure 3 As shown. The computer device includes a processor, a memory, a display screen, an input device, a network interface and a database connected through a system bus. Among them, the processor designed by the computer is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the corresponding data in this embodiment. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the above method is implemented.
[0072] Those skilled in the art will understand that Figure 3 The structure shown in is merely a block diagram of a portion of the structure related to the present application solution and does not constitute a limitation on the computer device to which the present application solution is applied.
[0073] The embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned lychee lossless sorting method is implemented. It can be understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.
[0074] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided in this application and used in the embodiments can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double-speed data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM.
[0075] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.
[0076] The above description is only a preferred embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.
Claims
1. A non-destructive sorting method for litchi, It is characterized in that include: Acquire a multispectral image of litchi, wherein the multispectral image includes at least two of a visible light image, an infrared light image, and an ultraviolet light image; Inputting the multispectral image into an image segmentation network for segmentation processing to obtain a valid image area of the litchi; Extracting target image features of the effective image area; The target image features are input into the trained prediction model, and the actual grade of the litchi is output through the prediction model.
2. The method for non-destructive sorting of litchi according to claim 1, It is characterized in that The step of extracting target image features from the effective image area comprises: Extracting initial image features of the effective image area; Target image features are screened out from the initial image features, wherein the number of the initial image features is greater than the number of the target image features.
3. The method for non-destructive sorting of litchi according to claim 1, It is characterized in that The step of inputting the multispectral image into an image segmentation network for segmentation processing to obtain a valid image area of the litchi comprises: Converting the multispectral image into a grayscale image; Binarizing the grayscale image to obtain a binary image; Performing a morphological operation on the binary image to obtain a morphological image, wherein the morphological operation includes at least one of dilation, erosion, opening operation and closing operation; The edge of the morphological image is extracted by an edge detection algorithm to obtain the effective image area of the litchi.
4. The method for non-destructive sorting of litchi according to claim 1, It is characterized in that The step of extracting target image features from the effective image area comprises: Separating the effective image area into RGB channels, extracting RGB image data of red, green and blue channels respectively, and performing statistical calculation on the RGB image data of each channel to extract color features of the effective image area as target image features, wherein the color features include at least one of the number of colors and their distribution ratio, color dispersion, color uniformity, brightness, average color and main color; and / or, Calculating the gray level co-occurrence matrix of the effective image area, and extracting the texture features of the effective image area as target image features, wherein the texture features include at least one of contrast, correlation, entropy and uniformity; and / or, Performing contour analysis on the effective image area to obtain geometric features of the effective image area as target image features, wherein the geometric features include at least one of shape, size and surface area; and / or, Performing spectral analysis on the effective image area to obtain spectral features of the effective image area as target image features, wherein the spectral features include at least one of reflectivity, refractive index and absorptivity.
5. The method for non-destructive sorting of litchi according to claim 1, It is characterized in that Before inputting the target image features into the trained prediction model and outputting the actual grade of the litchi through the prediction model, the method further includes: Pre-identifying and marking the grade of the litchi as a marked grade; Inputting the target image features into an initial prediction model; Analyzing the contribution of the target image features to the prediction result according to the marking level; Remove features with low contribution or negative impact on the prediction results, and retain features with high contribution to obtain the trained prediction model.
6. The method for non-destructive sorting of litchi according to claim 1, It is characterized in that The step of inputting the target image features into a trained prediction model and outputting the actual grade of the litchi through the prediction model comprises: Inputting the target image features into the trained prediction model to obtain the prediction level output by each decision tree in the prediction model; From the predicted grades, the predicted grades with the largest number are screened out to serve as the actual grades of the litchi.
7. The method for non-destructive sorting of litchi according to claim 1, It is characterized in that After inputting the target image features into the trained prediction model and outputting the actual grade of the litchi through the prediction model, the method further includes: The actual grade of the litchi is fed back in real time.
8. A non-destructive litchi sorting device, It is characterized in that include: An image acquisition module, used to acquire a multispectral image of litchi, wherein the multispectral image includes at least two of a visible light image, an infrared light image, and an ultraviolet light image; An image processing module, used for inputting the multispectral image into an image segmentation network for segmentation processing to obtain a valid image area of the litchi; A feature extraction module, used to extract target image features in the effective image area; The grade determination module is used to input the target image features into the trained prediction model and output the actual grade of the litchi through the prediction model.
9. A computer device comprising a memory and a processor, wherein the memory stores a computer program. It is characterized in that When the processor executes the computer program, the steps of the method for non-destructive sorting of litchis described in any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the method for non-destructive lychee sorting according to any one of claims 1 to 7 are implemented.
Citation Information
Patent Citations
Fruit quality damage-free detection method and system based on multiple spectral imaging technique
CN101059452A
Litchi disease automatic identification method and device based on hyperspectral image analysis processing method
CN113989639A
Hyperspectral detection method for ripeness of camellia oleifera fruits based on deep learning and sorting equipment
CN114112932A
Fruit grading method and device, electronic equipment, sorting system and storage medium
CN115713762A
Litchi phenotypic parameter detection method and system based on image processing and three-dimensional point cloud
CN118446969A
Cited By
Fruit and vegetable quality nondestructive testing imaging control system
CN119619026A
Tea quality detection method, system and equipment based on big data and medium
CN119672006A
A method, system, device, and medium for tea quality detection based on big data
CN119672006B
Real-time online detection method and system for fruit appearance defects
CN119845962A
