Litchi fruit quality detection method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of computer vision, and discloses a litchi fruit quality detection method and device, computer equipment and a storage medium, and the method comprises the steps: obtaining the variety of a to-be-detected litchi, and determining a specified excitation wavelength corresponding to the variety of the to-be-detected litchi; setting the excitation wavelength of fluorescence spectrum analysis equipment as the specified excitation wavelength; fluorescence spectrum data and multispectral image data of a plurality of different surface areas of the to-be-detected litchis are obtained respectively, the fluorescence spectrum data and the multispectral image data corresponding to the same surface areas are associated, and the sum of the different surface areas covers the whole surface of the to-be-detected litchis; inputting each group of associated fluorescence spectrum data and multispectral image data of the to-be-detected litchis into a pre-trained fruit quality prediction model, and outputting a quality degradation prediction result of the to-be-detected litchis; and rapid, accurate and lossless prediction of the quality condition of the litchi fruits is realized.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a litchi fruit quality detection method, device, computer equipment and storage medium. Background Art

[0002] Lychee fruit is highly favored by consumers for its unique flavor and rich nutrients. However, the ripening season of lychees is in the hot and rainy midsummer, which makes lychee fruit very easy to brown and rot after picking, quickly losing its original color, aroma and taste, greatly shortening its shelf life and limiting the long-distance transportation and long-term storage capacity of lychees.

[0003] Traditional methods of lychee quality testing mainly include sensory evaluation and physicochemical index testing. The sensory evaluation method relies on human vision, smell and taste to evaluate the quality of the fruit. Although this method is intuitive, it has the disadvantages of being time-consuming, labor-intensive, and the results are easily affected by subjective factors. In contrast, the physicochemical index testing method accurately measures the physical and chemical properties of lychees to judge their quality, which has the advantages of high accuracy and no interference from subjective factors. However, physicochemical testing usually requires the destruction of samples, which is complicated and time-consuming to operate, and it is difficult to meet the needs of large-scale rapid and non-destructive testing of lychees.

[0004] Therefore, how to effectively monitor and classify the quality of litchi during postharvest storage has become an important issue that needs to be addressed. Summary of the invention

[0005] Based on this, it is necessary to address the problem that the quality of existing litchi fruits is difficult to be rapidly and non-destructively tested during post-harvest storage, and a litchi fruit quality testing method, device, computer equipment and storage medium are proposed.

[0006] A first aspect of the present invention provides a method for detecting litchi fruit quality, the method comprising:

[0007] Obtaining the variety of litchi to be tested, and determining a designated excitation wavelength corresponding to the variety of litchi to be tested;

[0008] Setting the excitation wavelength of the fluorescence spectrum analysis device to the specified excitation wavelength;

[0009] Respectively obtaining fluorescence spectrum data and multispectral image data of several different surface areas of the litchi to be tested, and correlating the fluorescence spectrum data and multispectral image data corresponding to each same surface area, wherein the sum of the several different surface areas covers the entire surface of the litchi to be tested;

[0010] Each group of associated fluorescence spectrum data and multi-spectral image data of the litchi to be tested are input into a pre-trained fruit quality prediction model, and a quality degradation prediction result of the litchi to be tested is output.

[0011] Furthermore, the step of respectively acquiring the fluorescence spectrum data and the multispectral image data of several different surface areas of the litchi to be tested, and associating the fluorescence spectrum data and the multispectral image data corresponding to each same surface area comprises:

[0012] Acquire the associated data corresponding to a surface area, wherein the step of acquiring the associated data corresponding to a surface area comprises emitting excitation light of a specified excitation wavelength to a surface area of ​​the litchi to be tested through a fluorescence spectrum analysis device, recording a label value for labeling the surface area, acquiring fluorescence spectrum data of a specified sensitive band corresponding to the specified excitation wavelength, and acquiring multispectral image data taken by an image acquisition device, and storing the fluorescence spectrum data and the multispectral image data in association with the label value;

[0013] Determining whether data acquisition has been completed for all surface areas of the litchi to be tested;

[0014] If not, the position of the litchi to be tested is adjusted to a surface area where no data acquisition is performed, and the step of acquiring associated data corresponding to a surface area is continued.

[0015] Furthermore, the step of inputting each group of associated fluorescence spectrum data and multispectral image data of the litchi to be tested into a pre-trained fruit quality prediction model and outputting a prediction result of quality degradation of the litchi to be tested comprises:

[0016] Inputting the fluorescence spectrum data into the first input layer of the fruit quality prediction model, and inputting the multispectral image data into the second input layer of the fruit quality prediction model;

[0017] Extracting features from the fluorescence spectrum data through the first intermediate layer of the fruit quality prediction model to obtain spectrum features; and extracting features from the multispectral image data through the second intermediate layer of the fruit quality prediction model to obtain image features;

[0018] The spectral feature and the image feature are fused through the fusion layer of the fruit quality prediction model to obtain a fused feature vector;

[0019] The fused feature vector is input into the output layer of the fruit quality prediction model for feature analysis, and the quality degradation prediction result of the litchi to be tested is output, wherein the quality degradation prediction result includes a qualitative degradation prediction result and / or a quantitative degradation prediction result.

[0020] Furthermore, the step of fusing the spectral features with the image features through the fusion layer of the fruit quality prediction model to obtain a fused feature vector includes:

[0021] Performing feature fusion on each group of associated fluorescence spectrum data and multispectral image data of the litchi to be tested, respectively, to obtain sub-fusion feature vectors corresponding to each group of associated fluorescence spectrum data and multispectral image data;

[0022] Each group of the sub-fusion feature vectors is spliced, and the overlapping area is feature deleted according to the preset feature points on the multispectral image data to obtain the fusion feature vector.

[0023] Furthermore, the training step of the fruit quality prediction model includes:

[0024] Acquire a training data set, wherein the training data set includes a plurality of groups of associated and stored labeled fluorescence spectrum data and labeled multispectral image data, wherein the labeled content includes a quality degradation index; divide the data in the training data set into a training set and a validation set according to a preset ratio;

[0025] Constructing an initial prediction model, the initial prediction model includes a first input layer, a second input layer, a first intermediate layer, a second intermediate layer, a fusion layer and an output layer, the first input layer is connected to the first intermediate layer, the second input layer is connected to the second intermediate layer, the first intermediate layer and the second intermediate layer are connected to the fusion layer, and the fusion layer is connected to the output layer; wherein the first intermediate layer uses a long short-term memory network to extract spectral features, the second intermediate layer uses a convolutional neural network model to extract image features, and the fusion layer uses a multi-layer perceptron to perform feature fusion;

[0026] Inputting the training data in the training set into the initial prediction model, setting initial parameters, and performing training processing in a supervised learning manner to obtain a prediction model to be verified;

[0027] Inputting the verification data in the verification set into the prediction model to be verified for verification processing to obtain a verification result;

[0028] Determine whether the verification result is passed;

[0029] If so, the prediction model to be verified is used as the fruit quality prediction model.

[0030] Furthermore, before the steps of obtaining the variety of litchi to be tested and determining the designated excitation wavelength corresponding to the variety of litchi to be tested, the method further includes:

[0031] Record the variety of sampled litchi;

[0032] Measuring the surface of the sample litchi by three-dimensional fluorescence spectroscopy to obtain a three-dimensional fluorescence spectrum of the surface of the sample litchi;

[0033] Identifying a characteristic region corresponding to a predefined fruit characteristic from the three-dimensional fluorescence spectrum;

[0034] According to the characteristic area, the excitation wavelength and sensitive band that match the predefined fruit characteristics of the sample lychee are determined, and the matched laser wavelength is used as the designated excitation wavelength corresponding to the variety of the sample lychee, and the matched sensitive band is used as the designated sensitive band corresponding to the variety of the sample lychee.

[0035] Furthermore, after the step of inputting each group of associated fluorescence spectrum data and multispectral image data of the litchi to be tested into a pre-trained fruit quality prediction model and outputting the prediction result of the quality degradation of the litchi to be tested, the method further comprises:

[0036] Determining a matching litchi transportation destination according to the quality degradation prediction result;

[0037] The litchis to be tested are sorted and packaged according to the transportation destination.

[0038] A second aspect of the present invention provides a litchi fruit quality detection device, the device comprising:

[0039] A parameter acquisition module, used to acquire the variety of litchi to be tested, and determine a designated excitation wavelength corresponding to the variety of litchi to be tested;

[0040] A wavelength setting module, used to set the excitation wavelength of the fluorescence spectrum analysis device to the specified excitation wavelength;

[0041] A data acquisition module, for respectively acquiring fluorescence spectrum data and multispectral image data of a plurality of different surface areas of the litchi to be tested, and associating the fluorescence spectrum data and multispectral image data corresponding to each same surface area, wherein the sum of the plurality of different surface areas covers the entire surface of the litchi to be tested;

[0042] The quality degradation prediction module is used to input each group of associated fluorescence spectrum data and multi-spectral image data of the litchi to be tested into a pre-trained fruit quality prediction model, and output the quality degradation prediction result of the litchi to be tested.

[0043] A third aspect of the present invention provides a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor performs the following steps:

[0044] Obtaining the variety of litchi to be tested, and determining a designated excitation wavelength corresponding to the variety of litchi to be tested;

[0045] Setting the excitation wavelength of the fluorescence spectrum analysis device to the specified excitation wavelength;

[0046] Respectively obtaining fluorescence spectrum data and multispectral image data of several different surface areas of the litchi to be tested, and correlating the fluorescence spectrum data and multispectral image data corresponding to each same surface area, wherein the sum of the several different surface areas covers the entire surface of the litchi to be tested;

[0047] Each group of associated fluorescence spectrum data and multi-spectral image data of the litchi to be tested are input into a pre-trained fruit quality prediction model, and a quality degradation prediction result of the litchi to be tested is output.

[0048] A fourth aspect of the present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the following steps:

[0049] Obtaining the variety of litchi to be tested, and determining a designated excitation wavelength corresponding to the variety of litchi to be tested;

[0050] Setting the excitation wavelength of the fluorescence spectrum analysis device to the specified excitation wavelength;

[0051] Respectively obtaining fluorescence spectrum data and multispectral image data of several different surface areas of the litchi to be tested, and correlating the fluorescence spectrum data and multispectral image data corresponding to each same surface area, wherein the sum of the several different surface areas covers the entire surface of the litchi to be tested;

[0052] Each group of associated fluorescence spectrum data and multi-spectral image data of the litchi to be tested are input into a pre-trained fruit quality prediction model, and a quality degradation prediction result of the litchi to be tested is output.

[0053] The litchi fruit quality detection method, device, computer equipment and storage medium of the present invention obtain the variety of litchi to be tested and the corresponding specified excitation wavelength, set the excitation wavelength of the fluorescence spectrum analysis equipment, obtain the fluorescence spectrum data and multi-spectral image data of multiple areas on the surface of the litchi and associate them, and finally input them into a pre-trained fruit quality prediction model, thereby achieving rapid and accurate prediction of the quality status of the litchi fruit, improving the detection efficiency, reducing the subjectivity and error of human judgment, and maintaining the integrity and commercial value of the fruit, providing a basis for the long-distance transportation and storage preservation strategy of the litchi. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0055] in:

[0056] Figure 1 A diagram showing an application environment of a litchi fruit quality detection method in one embodiment;

[0057] Figure 2 A flowchart of a method for detecting litchi fruit quality in one embodiment;

[0058] Figure 3 is a structural block diagram of a litchi fruit quality detection device in one embodiment;

[0059] Figure 4 FIG. 4 is a structural block diagram of a computer device in one embodiment. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] Figure 1 FIG. 1 is a diagram of an application environment for litchi fruit quality detection in an embodiment. Figure 1 , the litchi fruit quality detection method is applied to a litchi fruit quality detection system. The litchi fruit quality detection system includes a terminal 110 and a server 120. The terminal 110 and the server 120 are connected via a network, the terminal 110 can be a desktop terminal or a mobile terminal, and the mobile terminal can be at least one of a mobile phone, a tablet computer, a laptop computer, etc. The server 120 can be implemented as an independent server or a server cluster composed of multiple servers. The terminal 110 is used to execute the steps of the litchi fruit quality detection method, and the server 120 is used to store pre-stored data used in the fruit quality prediction model, etc.

[0062] like Figure 2 As shown, in one embodiment, a method for detecting the quality of litchi fruit is provided. The method can be applied to both a terminal and a server. This embodiment is illustrated by applying to a terminal. The method for detecting the quality of litchi fruit specifically comprises the following steps:

[0063] S1: obtaining the variety of litchi to be tested, and determining a designated excitation wavelength corresponding to the variety of litchi to be tested;

[0064] S2: Setting the excitation wavelength of the fluorescence spectrum analysis device to the specified excitation wavelength;

[0065] S3: respectively obtaining fluorescence spectrum data and multispectral image data of several different surface areas of the litchi to be tested, and correlating the fluorescence spectrum data and multispectral image data corresponding to each same surface area, wherein the sum of the several different surface areas covers the entire surface of the litchi to be tested;

[0066] S4: inputting all groups of associated fluorescence spectrum data and multi-spectral image data of the litchi to be tested into a pre-trained fruit quality prediction model, and outputting a prediction result of quality degradation of the litchi to be tested.

[0067] In this embodiment, in the above steps S1-S2, the variety of litchi can be determined by the appearance characteristics of the litchi (such as skin color, fruit shape, size, etc.), or by comparison with an existing litchi variety database, and can be input after manual identification, or can be automatically identified by machine vision. The above-mentioned specified excitation wavelength is the optimal fluorescence excitation wavelength corresponding to the variety, which can be obtained by analyzing historical test data.

[0068] For a litchi to be tested, the above-mentioned designated laser wavelength can be one or multiple different values. When there are multiple values, after collecting the fluorescence spectrum data corresponding to one designated laser wavelength, it is necessary to set the next designated laser wavelength to collect the fluorescence spectrum data, that is, perform step S2 multiple times, and use the sum of the fluorescence spectrum data corresponding to the multiple designated laser wavelengths as the fluorescence spectrum data of step S3.

[0069] In the above step S3, a fluorescence spectrum analysis device and an image acquisition device are used to synchronously collect the litchi to be tested. The fluorescence spectrum analysis device is used to obtain the fluorescence spectrum data of the surface of the litchi fruit, and the image acquisition device is used to obtain the multi-spectral image data of the litchi surface, wherein the multi-spectral image data includes fluorescence image data and / or visible light image data. Multiple different areas on the litchi surface are collected, and the sum of these areas covers the entire litchi surface, thereby reflecting the overall quality of the litchi. During the collection process, the fluorescence spectrum data and the multi-spectral image data of each surface area are associated with each other to facilitate the implementation of subsequent steps. Exemplarily, this can be achieved by recording the position information of each area during collection, or by marking it with feature points.

[0070] In the above step S4, the above fruit quality prediction model is trained based on a large amount of fluorescence spectrum data and multispectral image data of litchi fruits with known quality conditions, and is used to accurately predict the quality status of litchi fruits, including whether quality degradation (such as browning, rot, etc.) occurs. After receiving the input data, the model will perform a series of calculations and analyses, and finally output the quality degradation prediction results of the litchi to be tested, such as the qualitative degradation grade of the fruit (such as excellent, slightly deteriorated, and severely deteriorated), quantitative degradation indicators (such as sugar loss rate, browning degree change rate), etc.

[0071] The litchi fruit quality detection method of the present embodiment obtains the variety of litchi to be tested and the corresponding specified excitation wavelength, sets the excitation wavelength of the fluorescence spectrum analysis equipment, obtains the fluorescence spectrum data and multi-spectral image data of multiple areas on the surface of the litchi and associates them, and finally inputs them into a pre-trained fruit quality prediction model, thereby achieving rapid and accurate prediction of the quality status of the litchi fruit, improving the detection efficiency, reducing the subjectivity and error of human judgment, and maintaining the integrity and commercial value of the fruit, providing an effective judgment basis for the long-distance transportation and storage preservation strategy of the litchi.

[0072] In a specific embodiment, the step S3 of respectively acquiring the fluorescence spectrum data and the multispectral image data of several different surface areas of the litchi to be tested, and associating the fluorescence spectrum data and the multispectral image data corresponding to each same surface area, comprises:

[0073] S301: Obtaining associated data corresponding to a surface area, wherein the step of obtaining associated data corresponding to a surface area comprises emitting excitation light of a specified excitation wavelength to a surface area of ​​the litchi to be tested through a fluorescence spectrum analysis device, recording a label value for marking the surface area, collecting fluorescence spectrum data of a specified sensitive band corresponding to the specified excitation wavelength, and collecting and obtaining multispectral image data taken by an image acquisition device, and storing the fluorescence spectrum data and the multispectral image data in association with the label value;

[0074] S302: Determine whether data acquisition has been completed for all surface areas of the litchi to be tested;

[0075] S303: If not, adjust the position of the litchi to be tested to a surface area where no data acquisition is performed, and continue to execute the step of acquiring associated data corresponding to a surface area.

[0076] In this embodiment, in the above step S301, the above label value is used to distinguish different surface areas of the litchi to be tested, for example, it can be a number, a letter, or a coordinate value containing position information. The fluorescence spectrum analysis device receives and records the fluorescence spectrum data reflected from the inside of the litchi fruit, and provides information on the changes in the chemical composition of the litchi fruit surface during storage, such as the loss of nutrients such as sugar, acidity, and vitamin C. The image acquisition device captures a multispectral image of the current surface area, providing intuitive information on the surface of the litchi fruit, such as whether there are quality problems such as browning and rot.

[0077] In the above steps S302-S303, it can be achieved by comparing the number of label values ​​of the collected data with the preset total number. If the number of label values ​​of the collected data is equal to the preset total number, or covers all surface areas in the preset list, it is determined that data acquisition has been completed for all surface areas. If the judgment result is no, that is, there are still surface areas for which data acquisition has not been performed, then the orientation of the litchi to be tested needs to be adjusted so that the next surface area to be collected is aligned with the fluorescence spectrum analysis device and the image acquisition device. Then, step S301 is repeated until data acquisition has been completed for all surface areas of the litchi to be tested. The orientation adjustment of the litchi to be tested can be performed manually or automatically flipped by a mechanical gripper, and this is not particularly limited in the embodiment of the present invention.

[0078] Furthermore, the collected multiple sets of related data are cleaned before being input into the fruit quality prediction model. Data cleaning includes removing outliers caused by equipment failure or operation errors, standardizing the fluorescence spectrum data (such as normalizing to the 0-1 interval) to eliminate the influence of intensity fluctuations on the model, and using Gaussian filtering or bilateral filtering to denoise the image data to reduce noise.

[0079] In a specific embodiment, the step S4 of inputting each group of associated fluorescence spectrum data and multispectral image data of the litchi to be tested into a pre-trained fruit quality prediction model and outputting the quality degradation prediction result of the litchi to be tested comprises:

[0080] S401: inputting the fluorescence spectrum data into the first input layer of the fruit quality prediction model, and inputting the multispectral image data into the second input layer of the fruit quality prediction model;

[0081] S402: extracting features from the fluorescence spectrum data through the first intermediate layer of the fruit quality prediction model to obtain spectrum features; and extracting features from the multispectral image data through the second intermediate layer of the fruit quality prediction model to obtain image features;

[0082] S403: fusing the spectral feature with the image feature through the fusion layer of the fruit quality prediction model to obtain a fusion feature vector;

[0083] S404: Input the fused feature vector to the output layer of the fruit quality prediction model for feature analysis, and output the quality degradation prediction result of the litchi to be tested, wherein the quality degradation prediction result includes a qualitative degradation prediction result and / or a quantitative degradation prediction result.

[0084] In this embodiment, in the above steps S401-S402, the fluorescence spectrum data is input through the first input layer, and feature extraction is performed through the first intermediate layer to identify spectral features related to litchi quality, such as key characteristic peaks (such as sensitive bands related to sugar) and their relative intensity, peak width, peak area and other parameters.

[0085] The multispectral image data is input through the second input layer, and feature extraction is performed through the second intermediate layer to identify image features related to the quality of litchi, such as texture features (such as fluorescence distribution uniformity), color features (such as fluorescence intensity attenuation in browning areas), and shape features (if there is a stalk). In a preferred embodiment, the fruit stalk is extracted as a special image feature. The presence or absence of the fruit stalk and its morphology are directly related to the overall quality of the litchi fruit. If the fruit stalk does not exist, the quality of the litchi fruit will deteriorate rapidly even if it is currently very fresh.

[0086] In the above steps S403-S404, the fused feature vector contains all the information in the two data types that is useful for litchi quality prediction. The fusion process combines the spectral features and image features into a unified fused feature vector, reduces feature redundancy through principal component analysis (PCA) or t-SNE dimensionality reduction method, and retains important features.

[0087] At the output layer, the model further analyzes and processes the fused feature vector to output the quality degradation prediction results of the tested litchi. These results may include qualitative degradation prediction results (e.g., fruit degradation grade: excellent, slightly deteriorated, severely deteriorated, etc.) and / or quantitative degradation prediction results (e.g., sugar loss rate, browning degree change rate, etc.). The above prediction results can provide valuable reference for the storage, transportation and sales of litchi.

[0088] In a specific embodiment, the step S403 of fusing the spectral features with the image features through the fusion layer of the fruit quality prediction model to obtain a fused feature vector includes:

[0089] S4031: performing feature fusion on each group of associated fluorescence spectrum data and multispectral image data of the litchi to be tested, respectively, to obtain sub-fusion feature vectors corresponding to each group of associated fluorescence spectrum data and multispectral image data;

[0090] S4032: Concatenate the sub-fusion feature vectors of each group, and perform feature deletion on the overlapping area according to the preset feature points on the multispectral image data to obtain the fusion feature vector.

[0091] In this embodiment, in the above step S4031, a feature fusion strategy (such as weighted summation, feature concatenation, principal component analysis, etc.) is used for each group of associated data to fuse each group of spectral features with the corresponding image features to form a sub-fusion feature vector.

[0092] In the above step S4032, all the sub-fusion feature vectors are spliced ​​into a long vector in a preset order (such as in the order of spectral data acquisition or image shooting) to form a preliminary fusion feature vector; in the spliced ​​preliminary fusion feature vector, the feature overlapping area caused by the image field of view is identified and marked, for example, it can be identified according to the above preset feature points, and the above preset feature points can be, for example, fixed markers in the image, specific parts of litchi fruits, etc., and their positional relationship in multiple images is used to determine whether the features overlap; feature deletion is performed on the identified overlapping area to remove redundant features and obtain a fusion feature vector.

[0093] In a specific embodiment, the training step of the fruit quality prediction model includes:

[0094] A1: Acquire a training data set, wherein the training data set includes a plurality of groups of associated and stored labeled fluorescence spectrum data and labeled multispectral image data, wherein the labeled content includes a quality degradation index; divide the data in the training data set into a training set and a validation set according to a preset ratio;

[0095] A2: construct an initial prediction model, the initial prediction model includes a first input layer, a second input layer, a first intermediate layer, a second intermediate layer, a fusion layer and an output layer, the first input layer is connected to the first intermediate layer, the second input layer is connected to the second intermediate layer, the first intermediate layer and the second intermediate layer are connected to the fusion layer, and the fusion layer is connected to the output layer; wherein the first intermediate layer uses a long short-term memory network to extract spectral features, the second intermediate layer uses a convolutional neural network model to extract image features, and the fusion layer uses a multi-layer perceptron to perform feature fusion;

[0096] A3: inputting the training data in the training set into the initial prediction model, setting initial parameters, and performing training processing in a supervised learning manner to obtain a prediction model to be verified;

[0097] A4: inputting the verification data in the verification set into the prediction model to be verified for verification processing to obtain a verification result;

[0098] A5: Determine whether the verification result is passed;

[0099] A6: If so, the prediction model to be verified is used as the fruit quality prediction model.

[0100] In this embodiment, in the above step A1, the above-mentioned quality degradation indicators include indicators of quality degradation during actual storage, such as sugar concentration, vitamin C content, browning degree, etc. The collected data is preprocessed, including data cleaning (removing noise, outliers, etc.), data standardization (converting data to the same scale), etc., to ensure the quality and consistency of the data. The preprocessed data set is divided according to a preset ratio (such as 80% training set, 20% validation set) for subsequent model training and validation.

[0101] In the above step A2, an initial prediction model is designed, which consists of a first input layer, a second input layer, a first intermediate layer (long short-term memory network LSTM), a second intermediate layer (convolutional neural network CNN), a fusion layer and an output layer. The first input layer receives fluorescence spectrum data, and a fully connected network is constructed through the first intermediate layer (LSTM) to perform multi-layer processing on the spectral features, extract nonlinear features, and capture the temporal dependencies in the spectral data. The second input layer receives multi-spectral image data, and extracts high-dimensional features of the fluorescence image through multi-layer convolution and pooling operations of the second intermediate layer (CNN). The fusion layer uses a multi-layer perceptron (MLP) to fuse the features extracted from the two intermediate layers to generate a fused feature vector. The output layer predicts the quality degradation index of litchi based on the fused feature vector.

[0102] In the above step A3, supervised learning is adopted. During the fast connection process, the back propagation algorithm is used to continuously adjust the model parameters. A dynamic learning rate adjustment strategy is introduced, such as learning rate decay or cosine annealing mechanism using an early stopping strategy to prevent overfitting.

[0103] In the above steps A4-A6, the model is verified and optimized through the validation set.

[0104] Specifically, the accuracy, precision, recall and F1 score are used to evaluate the classification performance during the model validation process. The mean square error (MSE), mean absolute error (MAE) and coefficient of determination (R 2 ) to evaluate the accuracy of the degradation rate prediction.

[0105] During the model optimization process, the attention mechanism is introduced to enhance the model's ability to focus on key features; transfer learning technology is used to fine-tune the pre-trained model to adapt to small sample scenarios; grid search or Bayesian optimization is used to automatically adjust parameters, such as adjusting the learning rate, network depth, and feature dimension; regularization technology (such as Dropout or L2 regularization) is introduced to suppress overfitting.

[0106] In addition, as data accumulates, the model is incrementally learned regularly to improve prediction accuracy.

[0107] Input the data in the validation set into the trained prediction model to be verified, make predictions, and calculate the model's performance indicators, such as accuracy, recall, F1 score, mean square error (MSE), etc., to evaluate the model's predictive ability and generalization performance. According to the preset performance indicator threshold or model performance evaluation standard, determine whether the verification result meets the requirements. If the verification result passes, that is, the model's performance indicator meets or exceeds the preset standard, the prediction model to be verified will be used as the final fruit quality prediction model for subsequent litchi quality prediction tasks. If the verification result fails, the model needs to be adjusted, such as modifying the model structure, adding data enhancement technology, adjusting parameters, etc., and retraining and verification until the requirements are met.

[0108] In a specific embodiment, before the step S1 of obtaining the variety of litchi to be tested and determining the designated excitation wavelength corresponding to the variety of litchi to be tested, the method further includes:

[0109] S01: record the variety of sample litchi;

[0110] S02: measuring the surface of the sample litchi by three-dimensional fluorescence spectroscopy to obtain a three-dimensional fluorescence spectrum of the surface of the sample litchi;

[0111] S03: identifying a characteristic region corresponding to a predefined fruit characteristic from the three-dimensional fluorescence spectrum;

[0112] S04: Determine, based on the characteristic area, an excitation wavelength and a sensitive band that match the predefined fruit characteristics of the sample lychee, and use the matched laser wavelength as the designated excitation wavelength corresponding to the variety of the sample lychee, and use the matched sensitive band as the designated sensitive band corresponding to the variety of the sample lychee.

[0113] This embodiment is used to determine the designated excitation wavelengths corresponding to litchi varieties that characterize different fruit characteristics. According to the known fruit characteristics of litchi (such as sugar content, acidity, vitamin C, etc.), the corresponding characteristic areas are found in the three-dimensional fluorescence spectrum. The spectrum is analyzed to identify the characteristic areas related to the fruit characteristics. In the characteristic area, the excitation wavelength and emission band with the highest or most representative fluorescence intensity are identified. The designated excitation wavelength and sensitive band information corresponding to each litchi variety are recorded and saved, and can be stored in a preset form so that matching can be performed from the preset form at the beginning of test step S1.

[0114] In a specific embodiment, the step S4 of inputting each group of associated fluorescence spectrum data and multispectral image data of the litchi to be tested into a pre-trained fruit quality prediction model and outputting the quality degradation prediction result of the litchi to be tested comprises:

[0115] S5: determining a matching litchi transportation destination according to the quality degradation prediction result;

[0116] S6: sorting and packing the litchis to be tested according to the transportation destination.

[0117] In this embodiment, in the above step S5, the quality degradation prediction result is converted into quantifiable quality indicators, such as freshness score, sugar content level, acidity range, etc. According to the correspondence between the preset transportation destination and the litchi quality index, the most suitable transportation destination is selected. The correspondence can be set in combination with factors such as transportation time and transportation conditions of different destinations. On the assembly line combined with the litchi fruit quality detection device of the embodiment of the present invention, sorting and packaging can be carried out directly on the assembly line, which greatly improves the efficiency and product quality of the litchi supply chain.

[0118] like Figure 3 As shown, in one embodiment, a litchi fruit quality detection device is provided, the device comprising:

[0119] The parameter acquisition module 10 is used to obtain the variety of litchi to be tested, and determine the designated excitation wavelength corresponding to the variety of litchi to be tested;

[0120] A wavelength setting module 20, used to set the excitation wavelength of the fluorescence spectrum analysis device to the specified excitation wavelength;

[0121] A data acquisition module 30 is used to respectively acquire the fluorescence spectrum data and multi-spectral image data of several different surface areas of the litchi to be tested, and associate the fluorescence spectrum data and multi-spectral image data corresponding to each same surface area, wherein the sum of the several different surface areas covers the entire surface of the litchi to be tested;

[0122] The quality degradation prediction module 40 is used to input each group of associated fluorescence spectrum data and multi-spectral image data of the litchi to be tested into a pre-trained fruit quality prediction model, and output a quality degradation prediction result of the litchi to be tested.

[0123] In a specific embodiment, the data acquisition module 30 includes:

[0124] A data acquisition unit, used for acquiring associated data corresponding to a surface area, wherein the step of acquiring associated data corresponding to a surface area comprises emitting excitation light of a specified excitation wavelength to a surface area of ​​the litchi to be tested through a fluorescence spectrum analysis device, recording a label value for labeling the surface area, acquiring fluorescence spectrum data of a specified sensitive band corresponding to the specified excitation wavelength, and acquiring multispectral image data taken by an image acquisition device, and storing the fluorescence spectrum data and the multispectral image data in association with the label value;

[0125] A judging unit, used for judging whether data acquisition has been completed for all surface areas of the litchi to be tested;

[0126] The adjustment unit is used to adjust the position of the litchi to be tested to a surface area where data has not been acquired if data acquisition has not been completed, and continue to execute the step of acquiring associated data corresponding to a surface area.

[0127] In a specific embodiment, the quality degradation prediction module 40 includes:

[0128] An input unit, used to input the fluorescence spectrum data into a first input layer of the fruit quality prediction model, and input the multispectral image data into a second input layer of the fruit quality prediction model;

[0129] A feature extraction unit, configured to extract features from the fluorescence spectrum data through the first intermediate layer of the fruit quality prediction model to obtain spectrum features; and to extract features from the multispectral image data through the second intermediate layer of the fruit quality prediction model to obtain image features;

[0130] A fusion unit, used for fusing the spectral feature with the image feature through a fusion layer of the fruit quality prediction model to obtain a fusion feature vector;

[0131] The output unit is used to input the fused feature vector into the output layer of the fruit quality prediction model for feature analysis, and output the quality degradation prediction result of the litchi to be tested, wherein the quality degradation prediction result includes a qualitative degradation prediction result and / or a quantitative degradation prediction result.

[0132] In a specific embodiment, the fusion unit comprises:

[0133] A fusion subunit is used to perform feature fusion on each group of associated fluorescence spectrum data and multispectral image data of the litchi to be tested, so as to obtain sub-fusion feature vectors corresponding to each group of associated fluorescence spectrum data and multispectral image data;

[0134] The splicing subunit is used to splice the sub-fusion feature vectors of each group and perform feature deletion on the overlapping area according to the preset feature points on the multispectral image data to obtain the fusion feature vector.

[0135] In a specific embodiment, the litchi fruit quality detection device includes a model training module, which is used to obtain a training data set, wherein the training data set includes multiple groups of associated stored labeled fluorescence spectrum data and labeled multispectral image data, and the labeled content includes a quality degradation index; the data in the training data set is divided into a training set and a validation set according to a preset ratio;

[0136] Constructing an initial prediction model, the initial prediction model includes a first input layer, a second input layer, a first intermediate layer, a second intermediate layer, a fusion layer and an output layer, the first input layer is connected to the first intermediate layer, the second input layer is connected to the second intermediate layer, the first intermediate layer and the second intermediate layer are connected to the fusion layer, and the fusion layer is connected to the output layer; wherein the first intermediate layer uses a long short-term memory network to extract spectral features, the second intermediate layer uses a convolutional neural network model to extract image features, and the fusion layer uses a multi-layer perceptron to perform feature fusion;

[0137] Inputting the training data in the training set into the initial prediction model, setting initial parameters, and performing training processing in a supervised learning manner to obtain a prediction model to be verified;

[0138] Inputting the verification data in the verification set into the prediction model to be verified for verification processing to obtain a verification result;

[0139] Determine whether the verification result is passed;

[0140] If so, the prediction model to be verified is used as the fruit quality prediction model.

[0141] In a specific embodiment, the litchi fruit quality detection device further includes:

[0142] In-bottle recording module, used to record the variety of sampled litchi;

[0143] A three-dimensional fluorescence data acquisition module is used to measure the surface of the sample litchi by three-dimensional fluorescence spectroscopy to obtain a three-dimensional fluorescence spectrum of the surface of the sample litchi;

[0144] A characteristic region determination module, used to identify a characteristic region corresponding to a predefined fruit characteristic from the three-dimensional fluorescence spectrum;

[0145] The designated excitation wavelength determination module is used to determine the excitation wavelength and sensitive band that match the predefined fruit characteristics of the sample lychee according to the characteristic area, and use the matched laser wavelength as the designated excitation wavelength corresponding to the variety of the sample lychee, and use the matched sensitive band as the designated sensitive band corresponding to the variety of the sample lychee.

[0146] In a specific embodiment, the litchi fruit quality detection device comprises:

[0147] A transportation destination determination module, used to determine a matching litchi transportation destination according to the quality degradation prediction result;

[0148] The sorting module is used to sort and pack the litchis to be tested according to the transportation destination.

[0149] In this embodiment, by obtaining the variety of litchi to be tested and the corresponding specified excitation wavelength, setting the excitation wavelength of the fluorescence spectrum analysis equipment, obtaining fluorescence spectrum data and multispectral image data of multiple areas on the surface of the litchi and correlating them, and finally inputting them into a pre-trained fruit quality prediction model, a rapid and accurate prediction of the quality of litchi fruit is achieved, the detection efficiency is improved, the subjectivity and error of human judgment are reduced, the integrity and commercial value of the fruit are maintained, and a basis is provided for the long-distance transportation and storage preservation strategy of litchi.

[0150] Figure 4 FIG. 1 shows an internal structure diagram of a computer device in an embodiment. The computer device may be a terminal or a server. Figure 4 As shown, the computer device includes a processor, a memory and a network interface connected via a system bus. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program. When the computer program is executed by the processor, the processor may implement the litchi fruit quality detection method. The internal memory may also store a computer program. When the computer program is executed by the processor, the processor may implement the litchi fruit quality detection method. Those skilled in the art will understand that Figure 4The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0151] In one embodiment, a computer device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the following steps:

[0152] Obtaining the variety of litchi to be tested, and determining a designated excitation wavelength corresponding to the variety of litchi to be tested;

[0153] Setting the excitation wavelength of the fluorescence spectrum analysis device to the specified excitation wavelength;

[0154] Respectively obtaining fluorescence spectrum data and multispectral image data of several different surface areas of the litchi to be tested, and correlating the fluorescence spectrum data and multispectral image data corresponding to each same surface area, wherein the sum of the several different surface areas covers the entire surface of the litchi to be tested;

[0155] Each group of associated fluorescence spectrum data and multi-spectral image data of the litchi to be tested are input into a pre-trained fruit quality prediction model, and a quality degradation prediction result of the litchi to be tested is output.

[0156] In this embodiment, by obtaining the variety of litchi to be tested and the corresponding specified excitation wavelength, setting the excitation wavelength of the fluorescence spectrum analysis equipment, obtaining fluorescence spectrum data and multispectral image data of multiple areas on the surface of the litchi and correlating them, and finally inputting them into a pre-trained fruit quality prediction model, a rapid and accurate prediction of the quality of litchi fruit is achieved, the detection efficiency is improved, the subjectivity and error of human judgment are reduced, the integrity and commercial value of the fruit are maintained, and a basis is provided for the long-distance transportation and storage preservation strategy of litchi.

[0157] In one embodiment, a computer-readable storage medium is provided, storing a computer program, wherein when the computer program is executed by a processor, the processor performs the following steps:

[0158] Obtaining the variety of litchi to be tested, and determining a designated excitation wavelength corresponding to the variety of litchi to be tested;

[0159] Setting the excitation wavelength of the fluorescence spectrum analysis device to the specified excitation wavelength;

[0160] Respectively obtaining fluorescence spectrum data and multispectral image data of several different surface areas of the litchi to be tested, and correlating the fluorescence spectrum data and multispectral image data corresponding to each same surface area, wherein the sum of the several different surface areas covers the entire surface of the litchi to be tested;

[0161] Each group of associated fluorescence spectrum data and multi-spectral image data of the litchi to be tested are input into a pre-trained fruit quality prediction model, and a quality degradation prediction result of the litchi to be tested is output.

[0162] In this embodiment, by obtaining the variety of litchi to be tested and the corresponding specified excitation wavelength, setting the excitation wavelength of the fluorescence spectrum analysis equipment, obtaining fluorescence spectrum data and multispectral image data of multiple areas on the surface of the litchi and correlating them, and finally inputting them into a pre-trained fruit quality prediction model, a rapid and accurate prediction of the quality of litchi fruit is achieved, the detection efficiency is improved, the subjectivity and error of human judgment are reduced, the integrity and commercial value of the fruit are maintained, and a basis is provided for the long-distance transportation and storage preservation strategy of litchi.

[0163] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing related hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct RAMbus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0164] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0165] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for detecting litchi fruit quality, characterized in that: The method comprises: Obtaining the variety of litchi to be tested, and determining a designated excitation wavelength corresponding to the variety of litchi to be tested; Setting the excitation wavelength of the fluorescence spectrum analysis device to the specified excitation wavelength; Respectively obtaining fluorescence spectrum data and multispectral image data of several different surface areas of the litchi to be tested, and correlating the fluorescence spectrum data and multispectral image data corresponding to each same surface area, wherein the sum of the several different surface areas covers the entire surface of the litchi to be tested; Each group of associated fluorescence spectrum data and multi-spectral image data of the litchi to be tested are input into a pre-trained fruit quality prediction model, and a quality degradation prediction result of the litchi to be tested is output.

2. The method for detecting litchi fruit quality according to claim 1, characterized in that: The step of respectively acquiring the fluorescence spectrum data and the multi-spectral image data of several different surface areas of the litchi to be tested, and associating the fluorescence spectrum data and the multi-spectral image data corresponding to each same surface area comprises: Acquire the associated data corresponding to a surface area, wherein the step of acquiring the associated data corresponding to a surface area comprises emitting excitation light of a specified excitation wavelength to a surface area of ​​the litchi to be tested through a fluorescence spectrum analysis device, recording a label value for labeling the surface area, acquiring fluorescence spectrum data of a specified sensitive band corresponding to the specified excitation wavelength, and acquiring multispectral image data taken by an image acquisition device, and storing the fluorescence spectrum data and the multispectral image data in association with the label value; Determining whether data acquisition has been completed for all surface areas of the litchi to be tested; If not, the position of the litchi to be tested is adjusted to a surface area where no data acquisition is performed, and the step of acquiring associated data corresponding to a surface area is continued.

3. The method for detecting litchi fruit quality according to claim 1, characterized in that: The step of inputting each group of associated fluorescence spectrum data and multispectral image data of the litchi to be tested into a pre-trained fruit quality prediction model and outputting a prediction result of quality degradation of the litchi to be tested comprises: Inputting the fluorescence spectrum data into the first input layer of the fruit quality prediction model, and inputting the multispectral image data into the second input layer of the fruit quality prediction model; Extracting features from the fluorescence spectrum data through the first intermediate layer of the fruit quality prediction model to obtain spectrum features; and extracting features from the multispectral image data through the second intermediate layer of the fruit quality prediction model to obtain image features; The spectral feature and the image feature are fused through the fusion layer of the fruit quality prediction model to obtain a fused feature vector; The fused feature vector is input into the output layer of the fruit quality prediction model for feature analysis, and the quality degradation prediction result of the litchi to be tested is output, wherein the quality degradation prediction result includes a qualitative degradation prediction result and / or a quantitative degradation prediction result.

4. The method for detecting litchi fruit quality according to claim 3, characterized in that: The step of fusing the spectral features with the image features through the fusion layer of the fruit quality prediction model to obtain a fused feature vector comprises: Performing feature fusion on each group of associated fluorescence spectrum data and multispectral image data of the litchi to be tested, respectively, to obtain sub-fusion feature vectors corresponding to each group of associated fluorescence spectrum data and multispectral image data; Each group of the sub-fusion feature vectors is spliced, and the overlapping area is feature deleted according to the preset feature points on the multispectral image data to obtain the fusion feature vector.

5. The method for detecting litchi fruit quality according to claim 1, characterized in that: The training steps of the fruit quality prediction model include: Acquire a training data set, wherein the training data set includes a plurality of groups of associated and stored labeled fluorescence spectrum data and labeled multispectral image data, wherein the labeled content includes a quality degradation index; divide the data in the training data set into a training set and a validation set according to a preset ratio; Constructing an initial prediction model, the initial prediction model includes a first input layer, a second input layer, a first intermediate layer, a second intermediate layer, a fusion layer and an output layer, the first input layer is connected to the first intermediate layer, the second input layer is connected to the second intermediate layer, the first intermediate layer and the second intermediate layer are connected to the fusion layer, and the fusion layer is connected to the output layer; wherein the first intermediate layer uses a long short-term memory network to extract spectral features, the second intermediate layer uses a convolutional neural network model to extract image features, and the fusion layer uses a multi-layer perceptron to perform feature fusion; Inputting the training data in the training set into the initial prediction model, setting initial parameters, and performing training processing in a supervised learning manner to obtain a prediction model to be verified; Inputting the verification data in the verification set into the prediction model to be verified for verification processing to obtain a verification result; Determine whether the verification result is passed; If so, the prediction model to be verified is used as the fruit quality prediction model.

6. The method for detecting litchi fruit quality according to claim 1, characterized in that: Before the steps of obtaining the variety of litchi to be tested and determining the designated excitation wavelength corresponding to the variety of litchi to be tested, the method further includes: Record the variety of sampled litchi; Measuring the surface of the sample litchi by three-dimensional fluorescence spectroscopy to obtain a three-dimensional fluorescence spectrum of the surface of the sample litchi; Identifying a characteristic region corresponding to a predefined fruit characteristic from the three-dimensional fluorescence spectrum; According to the characteristic area, the excitation wavelength and sensitive band that match the predefined fruit characteristics of the sample lychee are determined, and the matched laser wavelength is used as the designated excitation wavelength corresponding to the variety of the sample lychee, and the matched sensitive band is used as the designated sensitive band corresponding to the variety of the sample lychee.

7. The method for detecting litchi fruit quality according to claim 1, characterized in that: After the step of inputting all groups of associated fluorescence spectrum data and multi-spectral image data of the litchi to be tested into a pre-trained fruit quality prediction model and outputting the prediction result of quality degradation of the litchi to be tested, the method comprises: Determining a matching litchi transportation destination according to the quality degradation prediction result; The litchis to be tested are sorted and packaged according to the transportation destination.

8. A litchi fruit quality detection device, characterized in that: The device comprises: A parameter acquisition module, used to acquire the variety of litchi to be tested, and determine a designated excitation wavelength corresponding to the variety of litchi to be tested; A wavelength setting module, used to set the excitation wavelength of the fluorescence spectrum analysis device to the specified excitation wavelength; A data acquisition module, for respectively acquiring fluorescence spectrum data and multispectral image data of a plurality of different surface areas of the litchi to be tested, and associating the fluorescence spectrum data and multispectral image data corresponding to each same surface area, wherein the sum of the plurality of different surface areas covers the entire surface of the litchi to be tested; The quality degradation prediction module is used to input each group of associated fluorescence spectrum data and multi-spectral image data of the litchi to be tested into a pre-trained fruit quality prediction model, and output the quality degradation prediction result of the litchi to be tested.

9. A computer-readable storage medium, characterized in that: A computer program is stored, and when the computer program is executed by a processor, the processor executes the steps of the litchi fruit quality detection method according to any one of claims 1 to 7.

10. A computer device, characterized in that: The device includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the litchi fruit quality detection method according to any one of claims 1 to 7.

Images