Method and system for detecting mango ripeness based on raman spectroscopy

Abstract

The application provides a mango maturity detection method and system based on Raman spectrum, and belongs to the field of mango maturity detection. First, Raman spectrum data and image data of a mango surface are acquired; second, the Raman spectrum data are sequentially subjected to pretreatment operation, feature extraction operation, and are converted into a two-dimensional spectrum image; the two-dimensional spectrum image is input into a deformable expansion network, maturity prediction is performed, and a spectrum prediction result is obtained; then, according to the images of different regions of the mango surface, maturity prediction is performed, and a color prediction result is obtained; finally, the mango maturity is comprehensively predicted according to the spectrum prediction result and the color prediction result. The application realizes accurate prediction of the mango maturity by using the Raman spectrum data of the mango surface, extracting peak position and peak intensity features, applying the features to a convolutional neural network, and combining color auxiliary features; the application has non-destructive and high efficiency, and simultaneously provides a reliable technical means for the production and sales links of mangoes.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of mango ripeness detection, and specifically relates to a mango ripeness detection method and system based on Raman spectroscopy. BACKGROUND

[0002] In the process of planting and picking agricultural products, it is crucial to determine their ripeness. Mango, as a common tropical fruit, is widely planted and consumed around the world. However, traditional methods of mango ripeness detection have some limitations, such as high subjectivity, time and labor costs, and other issues.

[0003] Currently, some traditional methods of mango ripeness detection include visual inspection, fruit firmness measurement, and chemical analysis. Visual inspection mainly relies on observing the color, skin texture, and fruit size of mangoes to make judgments, but this method is easily affected by subjective factors and is difficult to accurately determine when the color change of mangoes is not obvious. Fruit firmness measurement can measure the hardness or elasticity of the fruit, but it requires destructive testing and causes damage to the quality of the mango. Chemical analysis methods include measuring parameters such as sugar content, acidity, and starch content in the fruit, but these methods require sampling and laboratory testing, which is time-consuming and not suitable for real-time detection.

[0004] To address the problems of traditional methods, some researchers have begun to explore the use of spectral technology to detect the ripeness of mangoes. Raman spectroscopy is a non-destructive spectral analysis technique that can provide information about molecular vibrations and structures. By scanning the Raman spectrum of mangoes, characteristic spectral images of the internal chemical composition of mangoes can be obtained, and the ripeness of mangoes can be determined.

[0005] However, existing methods and systems for detecting the ripeness of mangoes based on Raman spectroscopy still have some problems. For example, there are challenges in effectively extracting and analyzing spectral signals, and how to establish an accurate ripeness prediction model. Therefore, it is necessary to propose a method and system for detecting the ripeness of mangoes based on Raman spectroscopy to overcome the limitations of existing technology and improve the accuracy and efficiency of detection. SUMMARY

[0006] Based on the above technical problems, the present application provides a method and system for detecting the ripeness of mangoes based on Raman spectroscopy, which extracts Raman spectral features, establishes a ripeness prediction model, uses color as an auxiliary technology for joint prediction, and comprehensively analyzes and predicts the ripeness of mangoes to improve the accuracy of detection.

[0007] The present application provides a method for detecting the ripeness of mangoes based on Raman spectroscopy, which comprises:

[0008] Step S1: Obtain Raman spectrum data of different positions on the surface of the mango and images of different regions on the surface of the mango, wherein the different regions on the surface of the mango include a main-view root portion, a main-view middle portion, a main-view top portion, a rear-view root portion, a rear-view middle portion and a rear-view top portion;

[0009] Step S2: Preprocess the Raman spectrum data to obtain standard Raman spectrum data;

[0010] Step S3: Extract peak position and peak intensity features according to the standard Raman spectrum data, and convert them into a two-dimensional spectrum image;

[0011] Step S4: Input the two-dimensional spectrum image into a deformable expansion network to perform maturity prediction, and obtain a spectrum prediction result;

[0012] Step S5: Perform maturity prediction according to the images of the different regions on the surface of the mango to obtain a color prediction result;

[0013] Step S6: Integrate the spectrum prediction result and the color prediction result to comprehensively predict the maturity of the mango.

[0014] Optionally, the preprocessing of the Raman spectrum data to obtain the standard Raman spectrum data specifically comprises:

[0015] Performing filtering operation on the Raman spectrum data to obtain smoothed Raman spectrum data;

[0016] Performing baseline correction operation on the smoothed spectrum data to obtain stable Raman spectrum data;

[0017] Performing standardization operation on the stable spectrum data to obtain the standard Raman spectrum data.

[0018] Optionally, the inputting of the two-dimensional spectrum image into the deformable expansion network to perform maturity prediction to obtain the spectrum prediction result specifically comprises:

[0019] The deformable expansion network comprises a first standard convolution module, a first max-pooling layer, a deformable convolution module, an expansion convolution module, a second deformable convolution layer, a second standard convolution module, a third standard convolution module, a global average pooling layer, a full connection layer and a Softmax classifier;

[0020] The two-dimensional spectrum image is sequentially input into the first standard convolution module and the first max-pooling layer to perform standard convolution operation and max-pooling operation, and a feature map L3 is obtained;

[0021] The feature map L3 is input into the deformable convolution module to perform deformable convolution operation, and a feature map L9 is obtained;

[0022] The feature map L9 is input into the dilated convolution module for dilated convolution operation to obtain a feature map L19;

[0023] The feature map L19 is input into the second deformable convolution layer for deformable convolution operation to obtain a feature map L20;

[0024] The feature map L20 is sequentially input into the second standard convolution module and the third standard convolution module for standard convolution operation to obtain a feature map L24;

[0025] The feature map is sequentially input into the global average pooling layer, the full connection layer and the Softmax classifier for pooling, full connection and classification to obtain the spectral prediction result.

[0026] Optionally, the maturity prediction according to the images of different regions of the mango surface obtains a color prediction result, and specifically comprises:

[0027] The RGB value of the pixel in the image of the different regions of the mango surface is calculated;

[0028] The RGB value is judged with the color maturity range to obtain the color prediction result.

[0029] Optionally, the mango maturity is comprehensively predicted according to the spectral prediction result and the color prediction result, and specifically comprises:

[0030] The value of the mango maturity is calculated, and the specific formula is:

[0031] ML=α·V 光谱 +β·V 颜色

[0032] In the formula, ML is the value of the mango maturity, alpha is the weight corresponding to the spectrum, beta is the weight corresponding to the color, V 光谱 is the value corresponding to the spectral prediction result, and V 颜色 is the value corresponding to the color prediction result.

[0033] The mango maturity is determined according to the value of the mango maturity.

[0034] The application also provides a mango maturity detection system based on Raman spectrum, and the system comprises:

[0035] A mango data acquisition module is configured to acquire Raman spectrum data of different positions of a mango surface and images of different regions of the mango surface.

[0036] A data preprocessing module is configured to preprocess the Raman spectrum data to obtain standard Raman spectrum data.

[0037] a data feature extraction module configured to extract features from the standard Raman spectrum data, extract peak position and peak intensity features, and convert the features into a two-dimensional spectrum image;

[0038] a spectral maturity prediction module configured to input the two-dimensional spectrum image into a deformable expansion network, perform maturity prediction, and obtain a spectral prediction result;

[0039] a color maturity prediction module configured to perform maturity prediction based on images of different regions of the mango surface, and obtain a color prediction result;

[0040] a mango maturity prediction module configured to comprehensively predict the maturity of the mango based on the spectral prediction result and the color prediction result.

[0041] Optionally, the data preprocessing module specifically comprises:

[0042] a filtering submodule configured to perform filtering operation on the Raman spectrum data to obtain smooth Raman spectrum data;

[0043] a baseline correction submodule configured to perform baseline correction operation on the smooth spectrum data to obtain stable Raman spectrum data;

[0044] a normalization processing submodule configured to perform normalization operation on the stable spectrum data to obtain the standard Raman spectrum data.

[0045] Optionally, the spectral maturity prediction module specifically comprises:

[0046] a first standard convolution submodule configured to sequentially input the two-dimensional spectrum image into the first standard convolution module and the first max-pooling layer to perform standard convolution operation and max-pooling operation, and obtain a feature map L3;

[0047] a deformable convolution submodule configured to input the feature map L3 into the deformable convolution module to perform deformable convolution operation, and obtain a feature map L9;

[0048] an expansion convolution submodule configured to input the feature map L9 into the expansion convolution module to perform expansion convolution operation, and obtain a feature map L19;

[0049] a deformable convolution layer submodule configured to input the feature map L19 into the second deformable convolution layer to perform deformable convolution operation, and obtain a feature map L20;

[0050] a composite standard convolution submodule configured to sequentially input the feature map L20 into the second standard convolution module and the third standard convolution module to perform standard convolution operation, and obtain a feature map L24;

[0051] The pooling connection classification sub-module is used for sequentially inputting the feature map into the global average pooling layer, the full connection layer and the Softmax classifier for pooling, full connection and classification, so as to obtain the spectrum prediction result.

[0052] Optionally, the color maturity prediction module specifically comprises:

[0053] The numerical calculation sub-module is used for calculating the RGB numerical value of the pixel in the image of the different regions of the mango surface.

[0054] The judgment sub-module is used for judging the RGB numerical value and the color maturity range, so as to obtain the color prediction result.

[0055] Optionally, the mango maturity prediction module specifically comprises:

[0056] The mango maturity calculation sub-module is used for calculating the value of the mango maturity, and the specific formula is:

[0057] ML = alpha * V 光谱 + beta * V 颜色

[0058] In the formula, ML is the value of the mango maturity, alpha is the weight corresponding to the spectrum, beta is the weight corresponding to the color, V 光谱 is the value corresponding to the spectrum prediction result, and V 颜色 is the value corresponding to the color prediction result.

[0059] The maturity determination sub-module determines the mango maturity according to the value of the mango maturity.

[0060] Compared with the prior art, the present application has the following beneficial effects:

[0061] The maturity detection method based on Raman spectrum of the present application is non-destructive, and does not need destructive sample collection or processing of the mango. Compared with the traditional chemical analysis method, this non-destructive feature can maintain the integrity and quality of the mango, while improving the efficiency and real-time performance of the detection; the peak position and peak intensity characteristics are extracted by using the Raman spectrum technology, which can reflect the changes of the internal chemical composition of the mango. By analyzing and comparing the spectral characteristics of mangoes with different maturity, a convolutional neural network model is constructed to realize accurate prediction of the maturity of the mango; deformable convolution and dilated convolution are introduced, which can adaptively adjust the size and shape of the receptive field, so as to better capture the local and global features in the mango spectrum. This feature makes the prediction model better adapt to the feature changes of different mango samples, and improves the accuracy and robustness of the maturity prediction; in addition to the spectral characteristics, the present application also combines color auxiliary features for maturity prediction. The color of the mango usually changes significantly during the maturity change process, so combining color information with spectral characteristics can provide more comprehensive and reliable maturity prediction. This comprehensive use of multiple features can further improve the accuracy and reliability of the maturity detection. BRIEF DESCRIPTION OF DRAWINGS

[0062] Figure 1 A flowchart of the mango maturity detection method based on Raman spectrum of the present application;

[0063] Figure 2 A mango schematic diagram in the mango maturity detection method based on Raman spectrum of the present application;

[0064] Figure 3 A network structure diagram of the mango maturity detection method based on Raman spectrum of the present application;

[0065] Figure 4 A structure diagram of the mango maturity detection system based on Raman spectrum of the present application. DETAILED DESCRIPTION

[0066] The present application will be further described below in conjunction with specific implementation examples and drawings, but the present application is not limited to these examples.

[0067] Example 1

[0068] As shown in Figure 1 , the present application discloses a mango maturity detection method based on Raman spectrum, the method comprising:

[0069] Step S1: Obtain Raman spectrum data of different positions on the surface of the mango and images of different regions on the surface of the mango, wherein the different regions on the surface of the mango include the main view root, the main view middle, the main view top, the rear view root, the rear view middle and the rear view top.

[0070] Step S2: Preprocess the Raman spectrum data to obtain standard Raman spectrum data.

[0071] Step S3: Feature extraction is performed according to the standard Raman spectrum data, peak position and peak intensity features are extracted, and converted into a two-dimensional spectrum image.

[0072] Step S4: Input the two-dimensional spectrum image into the deformable expansion network to perform maturity prediction and obtain the spectrum prediction result.

[0073] Step S5: Perform maturity prediction according to the images of different regions on the surface of the mango to obtain the color prediction result.

[0074] Step S6: Comprehensive prediction of mango maturity according to the spectrum prediction result and the color prediction result.

[0075] The following will be discussed in detail:

[0076] Step S1: Obtain Raman spectrum data at different positions on the surface of the mango and images of different regions on the surface of the mango, including main view root, main view middle, main view top, rear view root, rear view middle and rear view top.

[0077] As shown in Figure 2 , step 1 specifically includes:

[0078] Step S11: Obtain Raman spectrum data at different positions on the surface of the mango using line scanning method, specifically including:

[0079] I. Prepare the scanning device, use a laser Raman spectrometer that can emit a laser beam and record the Raman spectrum reflected by the sample.

[0080] II. Determine the scanning path, select a path on the surface of the mango for scanning.

[0081] III. Set the scanning line, select the direction and starting point of the line scanning.

[0082] IV. Position the starting point: focus the laser on the starting point position on the scanning line.

[0083] V. Perform line scanning: starting from the starting point, move the laser Raman spectrometer at a constant speed along the scanning path, at each position, trigger the instrument to record the Raman spectrum data at that position.

[0084] VI. Collect Raman spectrum data: when the scanning device stays at each position, it will record the Raman spectrum data at that position. Raman spectrum is generated by Raman scattering in the laser beam reflected by the sample, which contains information about the molecular structure and chemical composition of the sample. As the laser moves, the Raman spectrometer records the spectrum data collected along the scanning line.

[0085] VII. Move to next position: move the scanning device to the next position and continue the line scan until all positions on the scan path are covered.

[0086] VIII. Complete scan line: when the laser moves to the end position of the scan line, the line scan is complete.

[0087] Step S12: Obtain images of the mango surface using a digital camera or image acquisition device. Ensure image clarity and color accuracy. Different regions of the mango surface include the main view root, main view middle, main view top, rear view root, rear view middle, and rear view top.

[0088] Step S2: Preprocess the Raman spectral data to obtain standard Raman spectral data, which specifically includes:

[0089] Step S21: Raman spectral data usually contains some random noise, which can interfere with the accuracy and reliability of the signal. To remove these noises, Gaussian filtering can be used to obtain smoothed Raman spectral data; filtering operation can smooth the spectral curve, reduce the influence of noise, enhance the signal-to-noise ratio and improve the accuracy of subsequent analysis.

[0090] Step S22: Perform baseline correction on the smoothed spectral data. Baseline drift may exist in Raman spectra, i.e., the overall trend of the spectral curve deviates from the true spectral characteristics. To eliminate the influence of baseline drift, baseline correction method can be used to adjust the spectral data, linear interpolation is used to correct the baseline, and the spectral curve is more accurate and stable, obtaining stable Raman spectral data.

[0091] Step S23: Perform standardization on the stable spectral data after baseline correction to obtain standard Raman spectral data. The original Raman spectral data is usually affected by different experimental conditions, instrument parameters, etc., resulting in differences in spectral intensity. To eliminate these differences, the spectral data can be processed by minimum-maximum standardization (discrete standardization) to eliminate intensity variations caused by sample thickness, laser power, or instrument settings. Standardization can scale the spectral data to the same range or distribution for subsequent feature extraction and analysis.

[0092] Step S3: Feature extraction based on standard Raman spectral data, extract peak position and peak intensity features, and convert to two-dimensional spectral image, which specifically includes:

[0093] Step S31: Extract peak intensity from Raman spectral data. For each position of the spectral data, find the peak value in the spectrum and extract the peak intensity as a feature. Peak value can be determined by Gaussian fitting. Peak intensity is the amplitude or intensity value of the peak, usually expressed as the peak value of spectral intensity.

[0094] Step S32: Raman spectrum data extracts peak position, for each position of the spectrum data, determine the wavelength position of the peak value appears, and extract the peak position as a feature. The peak position can be determined by finding the wavelength or wavenumber corresponding to the peak value.

[0095] Step S33: The extracted peak intensity is converted into an image, and the peak intensity features are rearranged into a 256x256 matrix. According to the collected spectrum data at different positions, the peak intensity feature values are filled into a 256x256 matrix according to the scanning order.

[0096] In this embodiment, the peak intensity value of the first position can be placed in the upper left corner of the matrix, the value of the second position can be placed in the right pixel position, and so on, until all the peak intensity values of the positions are filled into the matrix; the peak intensity matrix is normalized to map the peak intensity values to the gray value range of 0 to 255, and a linear normalization method is used to achieve this; the normalized peak intensity matrix is used as the gray channel of the image to generate a gray image. The gray value of each pixel represents the peak intensity value of the corresponding position, and the brightness of the pixel can be set according to the gray value.

[0097] Step S34: The extracted peak position is converted into an image, and the wavelength range is divided into three intervals according to the value of the peak position, and they are mapped to the red, green and blue components of the RGB channel.

[0098] In this embodiment, the shorter wavelength can be mapped to the blue channel according to the size of the wavelength or wavenumber, the middle wavelength can be mapped to the green channel, and the longer wavelength can be mapped to the red channel; the pixel value of the RGB channel is set according to the value of the peak position. For the peak position of the shorter wavelength, the pixel value of the blue channel is set to 255, and the pixel values of the green and red channels are set to 0. For the peak position of the middle wavelength, the pixel value of the green channel is set to 255, and the pixel values of the blue and red channels are set to 0. For the peak position of the longer wavelength, the pixel value of the red channel is set to 255, and the pixel values of the blue and green channels are set to 0.

[0099] Step S35: Merge the gray image and the image of the RGB channel to obtain a color image, take the gray image as the luminance channel of the image, and take the image of the RGB channel as the color channel; copy each pixel value of the gray image to the three channels of the color image to form a 256x256x3 color image; the pixel value in the luminance channel is determined by the gray image of the peak intensity, and the pixel value remains unchanged, while the pixel value in the color channel is determined by the peak position feature.

[0100] Step S36: Perform image enhancement, Gaussian filtering, and contrast and brightness adjustment on the color image to obtain a two-dimensional spectrum image.

[0101] Step S4: inputting the two-dimensional spectral image into the deformable dilation network to perform maturity prediction to obtain a spectral prediction result.

[0102] Figure 3 In the formula, Conv2D represents a standard convolution layer; Strides represents a stride; the normalized activation layer comprises a batch normalization layer (Batch Normalization) and an activation function layer (Activation (Relu)), and the normalized activation layer selects a Relu activation function; DilationConv2D represents a dilated convolution layer; Dense represents a fully connected layer; Convoffest2D represents a deformable convolution layer; GlobalAveragePooling2D represents a global average pooling layer; Maxpooling2D represents a maximum pooling layer; Add (μ, θ) represents element-wise addition of μ and θ; and Lσ represents each feature map obtained in the deformable dilation network, and σ is an integer in the range of [1, 24].

[0103] As shown in FIG. 4, step S4 specifically comprises: Figure 3

[0104] Step S41: sequentially inputting the two-dimensional spectral image into a first standard convolution module and a first maximum pooling layer to perform standard convolution operation and maximum pooling operation to obtain a feature map L3, specifically comprising:

[0105] The two-dimensional spectral image (256x256x3) is input into the first standard convolution layer to perform convolution operation to obtain a feature map L1, the first standard convolution layer has 12 convolution kernels, the size of the convolution kernel is 3x3, and the stride is 2; the feature map L1 is 128x128 with 12 channels; the feature map L1 is input into the first normalized activation layer to perform batch normalization and activation operation to obtain a feature map L2; the feature map L2 is 128x128 with 12 channels; and the feature map C2 is input into the first maximum pooling layer to perform maximum pooling operation to obtain a feature map L3; the first maximum pooling layer has a pooling window size of 3x3 and a stride of 2; and the feature map L3 is 64x64 with 12 channels.

[0106] In this embodiment, the first standard convolution module comprises a first standard convolution layer and a first normalized activation layer.

[0107] Step S42: inputting the feature map L3 into a deformable convolution module to perform deformable convolution operation to obtain a feature map L9, specifically comprising:

[0108] ​Step S421: input the feature map L3 into a first deformable convolution layer for deformable convolution operation to obtain a feature map L4, the first deformable convolution layer has 12 convolution kernel numbers and a convolution kernel size of 3*3 and a step of 1; the feature map L4 is 64*64 of 12 channels; input the feature map L4 into a second standard convolution layer for convolution operation to obtain a feature map L5, the second standard convolution layer has 48 convolution kernel numbers and a convolution kernel size of 3*3 and a step of 1; the feature map L5 is 64*64 of 48 channels; input the feature map L5 into a second normalization activation layer for batch normalization and activation operation to obtain a feature map L6; the feature map L6 is 64*64 of 48 channels.

[0109] Step S422: input the feature map L3 into a third standard convolution layer for convolution operation to obtain a feature map L7, the third standard convolution layer has 48 convolution kernel numbers and a convolution kernel size of 3*3 and a step of 1; the feature map L7 is 64*64 of 48 channels; input the feature map L7 into a third normalization activation layer for batch normalization and activation operation to obtain a feature map L8; the feature map L8 is 64*64 of 48 channels.

[0110] Step S423: input the feature map L6 and the feature map L8 into a first element-wise addition layer for element-wise addition operation to obtain a feature map L9; the feature map L9 is 64*64 of 48 channels.

[0111] In this embodiment, the deformable convolution module includes the first deformable convolution layer, the second standard convolution layer, the second normalization activation layer, the third standard convolution layer, the third normalization activation layer and the first element-wise addition layer.

[0112] Step S43: input the feature map L9 into a dilated convolution module for dilated convolution operation to obtain a feature map L19, which specifically includes:

[0113] Step S431: input the feature map L9 into a fourth standard convolution layer for convolution operation to obtain a feature map L10, the fourth standard convolution layer has 24 convolution kernel numbers and a convolution kernel size of 3*3 and a step of 1; the feature map L10 is 64*64 of 24 channels; input the feature map L10 into a fourth normalization activation layer for batch normalization and activation operation to obtain a feature map L11; the feature map L11 is 64*64 of 24 channels; input the feature map L11 into a fifth standard convolution layer for convolution operation to obtain a feature map L12, the fifth standard convolution layer has 96 convolution kernel numbers and a convolution kernel size of 3*3 and a step of 1; the feature map L12 is 32*32 of 96 channels; input the feature map L12 into a fifth normalization activation layer for batch normalization and activation operation to obtain a feature map L13; the feature map L13 is 32*32 of 96 channels.

[0114] Step S432: input the feature map L9 into a first dilated convolution layer for dilated convolution operation to obtain a feature map L14; the first dilated convolution layer has 48 convolution kernel numbers, a convolution kernel size of 3x3, a step of 1, and a dilation factor of 2; the feature map L14 is 64x64 with 48 channels; input the feature map L14 into a second normalization activation layer for batch normalization and activation operation to obtain a feature map L15; the feature map L15 is 64x64 with 48 channels; input the feature map L15 into a second max-pooling layer for max-pooling operation to obtain a feature map L16; the second max-pooling layer has a pooling window size of 3x3 and a step of 2; the feature map L16 is 32x32 with 48 channels; input the feature map L16 into a second dilated convolution layer for dilated convolution operation to obtain a feature map L17; the second dilated convolution layer has 96 convolution kernel numbers, a convolution kernel size of 3x3, a step of 1, and a dilation factor of 2; the feature map L17 is 32x32 with 96 channels; input the feature map L17 into a seventh normalization activation layer for batch normalization and activation operation to obtain a feature map L18; the feature map L18 is 32x32 with 96 channels.

[0115] Step S433: input the feature map L13 and the feature map L18 into a second element-wise addition layer for element-wise addition operation to obtain a feature map L19; the feature map L19 is 32x32 with 96 channels.

[0116] In this embodiment, the dilated convolution module includes a fourth standard convolution layer, a fourth normalization activation layer, a fifth standard convolution layer, a fifth normalization activation layer, a first dilated convolution layer, a sixth normalization activation layer, a second max-pooling layer, a second dilated convolution layer, a third normalization activation layer, and a second element-wise addition layer.

[0117] Step S44: input the feature map L19 into a second deformable convolution layer for deformable convolution operation to obtain a feature map L20; the second deformable convolution layer has 96 convolution kernel numbers and a convolution kernel size of 3x3 with a step of 1; the feature map L20 is 32x32 with 96 channels.

[0118] Step S45: input the feature map L20 into a second standard convolution module and a third standard convolution module in sequence for standard convolution operation to obtain a feature map L24, which specifically includes:

[0119] input the feature map L20 into a sixth standard convolution layer for convolution operation to obtain a feature map L21; the sixth standard convolution layer has 192 convolution kernel numbers, a convolution kernel size of 3x3, and a step of 2; the feature map L21 is 16x16 with 192 channels; input the feature map L21 into an eighth normalization activation layer for batch normalization and activation operation to obtain a feature map L22; the feature map L22 is 16x16 with 192 channels.

[0120] The feature map L22 is input into a seventh standard convolution layer for convolution operation to obtain a feature map L23, the seventh standard convolution layer has 384 convolution kernel numbers and a convolution kernel size of 3*3 and a step of 1; the feature map L23 is 64*64 with 384 channels; the feature map L23 is input into a ninth normalization activation layer for batch normalization and activation operation to obtain a feature map L24; the feature map L24 is 8*8 with 384 channels.

[0121] In the embodiment, the second standard convolution module includes a sixth standard convolution layer and an eighth normalization activation layer; and the third standard convolution module includes a seventh standard convolution layer and a ninth normalization activation layer.

[0122] Step S46: The feature map is sequentially input into a global average pooling layer, a full connection layer and a Softmax classifier for pooling, full connection and classification to obtain a spectrum prediction result, the spectrum prediction result including one of four ripeness degrees of mature, semi-mature, over-mature and unripe.

[0123] In the embodiment, the deformable expansion network includes a first standard convolution module, a first maximum pooling layer, a deformable convolution module, an expansion convolution module, a second deformable convolution layer, a second standard convolution module, a third standard convolution module, a global average pooling layer, a full connection layer and a Softmax classifier.

[0124] Step S5: The ripeness degree is predicted according to the images of different regions on the surface of the mango to obtain a color prediction result, as shown in FIG. 5, and the color prediction result specifically includes: Figure 2

[0125] Step S51: The RGB values of pixels in the images of different regions on the surface of the mango are calculated, and the calculation specifically includes:

[0126] (1) The images of different regions on the surface of the collected mango are preprocessed, including image denoising and contrast enhancement operation, so as to improve the accuracy of subsequent color acquisition.

[0127] (2) The image segmentation technology is used to separate the mango surface region from the background, and the threshold segmentation and edge detection method can be used.

[0128] (3) In the mango surface region, the color information of each pixel in the main view root, the main view middle, the main view top, the rear view root, the rear view middle and the rear view top is calculated to obtain the color data of the mango, and the RGB color space is used for color representation.

[0129] Step S52: The RGB values are judged with the color ripeness range to obtain a color prediction result, and the judgment specifically includes:

[0130] ​First, the acquired mango color data is calculated, and the proportion of the color falling within the range in each area (root, middle and top) is calculated. The proportion is obtained by dividing the number of color values falling within the range by the total number of color values.

[0131] In this embodiment, a mango image is collected, and the main view can be divided into three regions: root, middle and top. For each region, the pixels in the region need to be traversed, and it is determined whether the color value of each pixel is within the corresponding color range. Assuming that the range of red color of mature mango root is [50, 100], the range of red color of middle is [101, 150], and the range of red color of top is [151, 200].

[0132] For the root region, the pixels in the root region are traversed, and the number of pixels with red color values falling within the range [50, 100] is counted. The number of pixels falling within the range of mature mango is divided by the total number of pixels. Similarly, the number of pixels falling within the red color range in the middle region and the top region can be counted.

[0133] In this embodiment, the color range corresponding to the mature mango is set as root R: [150, 255], G: [0, 100], B: [0, 50], middle R: [200, 255], G: [150, 200], B: [0, 100], and top R: [200, 255], G: [150, 220], B: [0, 50]; the color range corresponding to the semi-mature mango is set as root R: [150, 200], G: [100, 150], B: [0, 50], middle R: [150, 200], G: [150, 200], B: [0, 100], and top R: [200, 255], G: [150, 220], B: [0, 50]; the color range corresponding to the over-mature mango is set as root R: [150, 255], G: [0, 100], B: [0, 50], middle R: [150, 200], G: [100, 150], B: [0, 100], and top R: [200, 255], G: [150, 220], B: [0, 50]; and the color range corresponding to the unripe mango is set as root R: [0, 100], G: [100, 150], B: [0, 50], middle R: [100, 150], G: [150, 200], B: [0, 100], and top R: [150, 200], G: [150, 220], B: [0, 50]. The color range in the present application is set according to the actual mango variety, light conditions and environmental influences.

[0134] Second, according to the proportion of the maturity range occupied by each pixel in each region, analysis is performed, and the corresponding maturity with the largest proportion is selected as the color prediction result according to the size of the proportion from large to small.

[0135] Step S6: The mango maturity is comprehensively predicted according to the spectral prediction result and the color prediction result, and specifically includes:

[0136] Step S61: The value of the mango maturity is calculated, and the specific formula is:

[0137] ML = a · V 光谱 + b · V 颜色

[0138] In the formula, ML is the value of the mango maturity, a is the weight corresponding to the spectrum, b is the weight corresponding to the color, V 光谱 is the value corresponding to the spectral prediction result, V 颜色 is the value corresponding to the color prediction result, and the value of a+b is a fixed value 1. In the present application, a is set to 0.7, and b is set to 0.3. The values of a and b are set according to actual conditions.

[0139] In the present embodiment, the values corresponding to unripe, semi-ripe, ripe and overripe are 1, 2, 3 and 4, the spectral prediction result is one of ripe, semi-ripe, overripe and unripe, and the color prediction result is one of ripe, semi-ripe, overripe and unripe.

[0140] Step S62: The mango maturity is determined according to the value of the mango maturity, and specifically includes:

[0141] When the value of the mango maturity is greater than or equal to the first threshold value, the current mango maturity is overripe.

[0142] When the value of the mango maturity is greater than or equal to the second threshold value and less than the first threshold value, the current mango maturity is ripe.

[0143] When the value of the mango maturity is greater than or equal to the third threshold value and less than the second threshold value, the current mango maturity is semi-ripe.

[0144] When the value of the mango maturity is greater than or equal to the fourth threshold value and less than the third threshold value, the current mango maturity is unripe.

[0145] The mango maturity includes unripe, semi-ripe, ripe and overripe.

[0146] The first threshold value, the second threshold value, the third threshold value and the fourth threshold value can be set according to actual needs.

[0147] Embodiment 2

[0148] For example, Figure 4As shown, the mango maturity detection system based on Raman spectrum is disclosed, the system comprises:

[0149] The mango data acquisition module 10 is used for acquiring Raman spectrum data of different positions of the mango surface and images of different regions of the mango surface.

[0150] The data preprocessing module 20 is used for preprocessing the Raman spectrum data to obtain standard Raman spectrum data.

[0151] The data feature extraction module 30 is used for extracting features according to the standard Raman spectrum data, extracting peak position and peak intensity features, and converting into a two-dimensional spectrum image.

[0152] The spectrum maturity prediction module 40 is used for inputting the two-dimensional spectrum image into a deformable expansion network to perform maturity prediction, and obtaining a spectrum prediction result.

[0153] The color maturity prediction module 50 is used for performing maturity prediction according to the images of different regions of the mango surface to obtain a color prediction result.

[0154] The mango maturity prediction module 60 is used for comprehensively predicting the mango maturity according to the spectrum prediction result and the color prediction result.

[0155] As an optional implementation, the data preprocessing module 20 of the present application specifically comprises:

[0156] The filtering submodule is used for filtering the Raman spectrum data to obtain smooth Raman spectrum data;

[0157] The baseline correction submodule is used for performing baseline correction on the smooth spectrum data to obtain stable Raman spectrum data.

[0158] The standardization processing submodule is used for performing standardization on the stable spectrum data to obtain standard Raman spectrum data.

[0159] As an optional implementation, the spectrum maturity prediction module 40 of the present application specifically comprises:

[0160] The first standard convolution submodule is used for sequentially inputting the two-dimensional spectrum image into the first standard convolution module and the first maximum pooling layer to perform standard convolution operation and maximum pooling operation, and obtaining a feature map L3.

[0161] The deformable convolution submodule is used for inputting the feature map L3 into the deformable convolution module to perform deformable convolution operation, and obtaining a feature map L9.

[0162] The expansion convolution submodule is used for inputting the feature map L9 into the expansion convolution module to perform expansion convolution operation, and obtaining a feature map L19.

[0163] A deformable convolution layer submodule is configured to input the feature map L19 into a second deformable convolution layer to perform a deformable convolution operation, so as to obtain a feature map L20.

[0164] A composite standard convolution submodule is configured to input the feature map L20 into a second standard convolution module and a third standard convolution module in sequence to perform a standard convolution operation, so as to obtain a feature map L24.

[0165] A pooling connection classification submodule is configured to input the feature map into a global average pooling layer, a full connection layer and a Softmax classifier in sequence to perform pooling, full connection and classification, so as to obtain a spectral prediction result.

[0166] As an optional implementation, the color maturity prediction module 50 specifically comprises:

[0167] A numerical calculation submodule is configured to calculate RGB values of pixels in images of different regions on the surface of the mango.

[0168] A judgment submodule is configured to judge the RGB values against a color maturity range, so as to obtain a color prediction result.

[0169] As an optional implementation, the mango maturity prediction module 60 specifically comprises:

[0170] A mango maturity calculation submodule is configured to calculate a value of the mango maturity, and the specific formula is:

[0171] ML = α · V 光谱 + β · V 颜色

[0172] In the formula, ML is the value of the mango maturity, α is a weight corresponding to the spectrum, β is a weight corresponding to the color, V 光谱 is a value corresponding to the spectral prediction result, and V 颜色 is a value corresponding to the color prediction result.

[0173] A maturity determination submodule is configured to determine the mango maturity according to the value of the mango maturity.

[0174] The above merely describes the preferred embodiments of the present application and is not used to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for detecting mango maturity based on Raman spectroscopy, characterized in that, The method includes: Step S1: Obtain Raman spectral data at different locations on the mango surface and images of different regions on the mango surface, including the root of the main view, the middle of the main view, the top of the main view, the root of the rear view, the middle of the rear view, and the top of the rear view. Step S2: Preprocess the Raman spectral data to obtain standard Raman spectral data; Step S3: Perform feature extraction based on the standard Raman spectral data, extract peak position and peak intensity features, and convert them into a two-dimensional spectral image; Step S4: Input the two-dimensional spectral image into a deformable dilatation network to predict maturity and obtain the spectral prediction result, specifically including: The deformable dilatational network includes a first standard convolutional module, a first max pooling layer, a deformable convolutional module, a dilatational convolutional module, a second deformable convolutional layer, a second standard convolutional module, a third standard convolutional module, a global average pooling layer, a fully connected layer, and a Softmax classifier; The two-dimensional spectral image is sequentially input into the first standard convolution module and the first max pooling layer to perform standard convolution and max pooling operations, resulting in feature map L3; The feature map L3 is input into the deformable convolution module to perform deformable convolution operation, resulting in feature map L9; The feature map L9 is input into the dilated convolution module to perform a dilated convolution operation, resulting in feature map L19; The feature map L19 is input into the second deformable convolutional layer for deformable convolution operation to obtain the feature map L20. The feature map L20 is sequentially input into the second standard convolution module and the third standard convolution module to perform standard convolution operations, resulting in feature map L24; The feature map is sequentially input into the global average pooling layer, the fully connected layer, and the Softmax classifier for pooling, full connection, and classification to obtain the spectral prediction result. Step S5: Based on the images of different areas on the surface of the mango, predict the ripeness and obtain the color prediction result; Step S6: Predict the mango ripeness based on the combined results of the spectral prediction and the color prediction.

2. The method for detecting mango maturity based on Raman spectroscopy according to claim 1, characterized in that, The step of preprocessing the Raman spectral data to obtain standard Raman spectral data specifically includes: The Raman spectral data is filtered to obtain smoothed Raman spectral data; The smoothed Raman spectral data are subjected to baseline correction to obtain stable Raman spectral data; The stable Raman spectral data is standardized to obtain the standard Raman spectral data.

3. The method for detecting mango maturity based on Raman spectroscopy according to claim 1, characterized in that, The step of predicting ripeness based on images of different regions of the mango surface to obtain color prediction results specifically includes: Calculate the RGB values ​​of pixels in images of different regions on the surface of the mango; The color prediction result is obtained by comparing the RGB values ​​with the color maturity range.

4. The method for detecting mango maturity based on Raman spectroscopy according to claim 1, characterized in that, The method of predicting mango ripeness based on the combined spectral prediction results and color prediction results specifically includes: The specific formula for calculating the ripeness of a mango is as follows: ； In the formula, This represents the ripeness value of the mango. The weights corresponding to the spectra. The weight corresponding to the color, The value corresponding to the spectral prediction result. This represents the value corresponding to the color prediction result; The maturity of a mango is determined based on its maturity value.

5. A mango maturity detection system based on Raman spectroscopy, characterized in that, The system includes: The mango data acquisition module is used to acquire Raman spectral data at different locations on the mango surface and images of different regions on the mango surface. The data preprocessing module is used to preprocess the Raman spectral data to obtain standard Raman spectral data; The data feature extraction module performs feature extraction based on the standard Raman spectral data, extracting peak position and peak intensity features, and converting them into a two-dimensional spectral image; The spectral maturity prediction module is used to input the two-dimensional spectral image into a deformable dilatational network to predict maturity and obtain spectral prediction results, specifically including: The first standard convolutional submodule is used to sequentially input the two-dimensional spectral image into the first standard convolutional module and the first max pooling layer to perform standard convolution and max pooling operations to obtain feature map L3; A deformable convolution submodule is used to input the feature map L3 into the deformable convolution module to perform deformable convolution operation to obtain feature map L9; The dilated convolution submodule is used to input the feature map L9 into the dilated convolution module to perform dilated convolution operation to obtain the feature map L19. The deformable convolutional layer submodule is used to input the feature map L19 into the second deformable convolutional layer to perform deformable convolution operation to obtain the feature map L20. The composite standard convolution submodule is used to sequentially input the feature map L20 into the second standard convolution module and the third standard convolution module to perform standard convolution operations, thereby obtaining the feature map L24; The pooling-connection-classification submodule is used to sequentially input the feature map into a global average pooling layer, a fully connected layer, and a Softmax classifier for pooling, full connection, and classification to obtain the spectral prediction result. The color maturity prediction module predicts maturity based on images of different areas on the mango surface and obtains the color prediction result. The mango maturity prediction module predicts mango maturity based on the combined results of the spectral prediction and the color prediction.

6. The mango maturity detection system based on Raman spectroscopy according to claim 5, characterized in that, The data preprocessing module specifically includes: The filtering submodule is used to filter the Raman spectral data to obtain smoothed Raman spectral data. The baseline correction submodule is used to perform baseline correction on the smoothed Raman spectral data to obtain stable Raman spectral data. The standardization processing submodule is used to standardize the stable Raman spectral data to obtain the standard Raman spectral data.

7. The mango maturity detection system based on Raman spectroscopy according to claim 5, characterized in that, The color maturity prediction module specifically includes: The numerical calculation submodule is used to calculate the RGB values ​​of pixels in images of different regions on the surface of the mango. The judgment submodule is used to judge the RGB values ​​and the color maturity range to obtain the color prediction result.

8. The mango maturity detection system based on Raman spectroscopy according to claim 5, characterized in that, The mango maturity prediction module specifically includes: The mango maturity calculation submodule is used to calculate the maturity value of mangoes. The specific formula is as follows: ； In the formula, This represents the ripeness value of the mango. The weights corresponding to the spectra. The weight corresponding to the color, The value corresponding to the spectral prediction result. This represents the value corresponding to the color prediction result; The maturity determination submodule determines the maturity of the mango based on the mango maturity value.

Images