Fruit maturity classification method and system based on vein information

Abstract

The invention discloses a fruit maturity classification method and system based on vein information, and relates to the technical field of image processing, and the method comprises the steps: firstly obtaining a plurality of fruit surface pictures of fruits with different maturity, carrying out the mask processing, obtaining corresponding fruit vein pictures, forming an image pair, and constructing a pre-training data set; information which is not fruit vein information but is highly related to fruit maturity in the fruit surface picture can be learned; constructing an encoder pre-training module, and training the encoder pre-training module by using the pre-training data set to obtain a trained encoder; and constructing a fruit maturity classification model based on the trained encoder, training the fruit maturity classification model by using a plurality of fruit surface pictures to obtain the trained fruit maturity classification model, and inputting the surface picture of the to-be-classified fruit to obtain the maturity category of the to-be-classified fruit. According to the method, the vein information in the fruit surface picture can be effectively utilized, accurate identification and classification of the fruit maturity can be realized, and the robustness and the generalization ability are high.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and more specifically, to a method and system for classifying fruit maturity based on vein information. Background Art

[0002] The maturity of fruits is a key factor determining their quality, taste, and nutritional value. Traditional methods for judging fruit maturity mainly rely on manual experience, such as observing characteristics such as the appearance color, texture, and size of the fruits. However, manual judgment has the following disadvantages: 1. Strong subjectivity: Different people may have different criteria for judging maturity, resulting in inconsistent judgment results; 2. Low efficiency: Manual judgment requires observing fruits one by one, with low efficiency and difficulty in meeting the needs of large-scale fruit detection; 3. Poor accuracy: Manual judgment is easily affected by environmental factors, personal experience, etc., resulting in inaccurate judgment results.

[0003] In recent years, with the development of computer vision and deep learning technologies, image-based methods for identifying fruit maturity have gradually emerged. These methods are mainly divided into two categories: 1. Methods based on the overall appearance characteristics of fruits: Using characteristics such as the color, texture, and shape of fruit images for maturity identification. Such methods are simple and easy to implement, but are easily affected by factors such as lighting and background, resulting in low identification accuracy; 2. Methods based on local characteristics of fruits: Using specific local regions in fruit images, such as small holes and spots on the fruit surface, for maturity identification. Such methods can effectively reduce the influence of environmental factors, but require a large amount of labeled data and are difficult to generalize to different fruit species. Summary of the Invention

[0004] One of the purposes of the present invention is to provide a method for classifying fruit maturity based on vein information to overcome the defects of the above-mentioned prior art that rely on a large amount of labeled data and cannot accurately and efficiently identify the maturity of different fruits; the second purpose is to provide a system for classifying fruit maturity based on vein information.

[0005] To solve the above technical problems, the technical solution of the present invention is as follows:

[0006] The present invention provides a method for classifying fruit maturity based on vein information, including:

[0007] S1: Obtain a number of fruit surface pictures of fruits with different maturities, and perform masking processing on each of the fruit surface pictures to obtain corresponding fruit vein pictures;

[0008] S2: Combine the corresponding fruit surface pictures and fruit vein pictures into image pairs to construct a pre-training data set;

[0009] S3: Construct an encoder pre-training module, including an encoder, an information extractor, and a cross-correlation matrix learner; use the pre-training dataset to train the encoder pre-training module to obtain a trained encoder;

[0010] S4: Based on the trained encoder, construct a fruit maturity classification model, and use the several fruit surface images to train the fruit maturity classification model to obtain a trained fruit maturity classification model;

[0011] S5: Obtain the surface image of the fruit to be classified, input it into the trained fruit maturity classification model, and obtain the maturity category of the fruit to be classified.

[0012] Preferably, the masking process includes using a binary mask to cover the non-vein part in the fruit surface image to obtain a fruit vein image.

[0013] It should be noted that the veins presented on the fruit surface are an important part of the fruit structure, and their distribution, color, thickness and other characteristics are closely related to the maturity. The veins of mature fruits are more evenly distributed, while the veins of immature fruits are relatively concentrated; the veins of mature fruits are darker in color, while the veins of immature fruits are lighter in color; the veins of mature fruits are thicker, while the veins of immature fruits are thinner. The selected fruit varieties with different maturities are fruits with obvious vein characteristics. Different maturities include immature, mature, and overripe. The vein information maps the characteristics of different maturity stages of the fruit. By extracting and analyzing the fruit vein information as an important feature, accurate identification and classification of fruit maturity can be achieved.

[0014] Preferably, the encoder includes an initial convolutional layer, a max pooling layer, a first-stage unit, a second-stage unit, a third-stage unit, a fourth-stage unit, and an average pooling layer connected in sequence.

[0015] Preferably, the first-stage unit includes a first residual block, a second residual block, and a third residual block connected in sequence;

[0016] The second-stage unit includes a fourth residual block, a fifth residual block, a sixth residual block, and a seventh residual block connected in sequence;

[0017] The third-stage unit includes an eighth residual block, a ninth residual block, a tenth residual block, an eleventh residual block, a twelfth residual block, and a thirteenth residual block connected in sequence;

[0018] The fourth-stage unit includes a fourteenth residual block, a fifteenth residual block, and a sixteenth residual block connected in sequence.

[0019] Preferably, the information extractor includes a first fully-connected layer, a first batch normalization layer, a first activation function layer, a second fully-connected layer, a second batch normalization layer, a second activation function layer, and a third fully-connected layer that are connected in sequence.

[0020] Preferably, training the encoder pre-training module using the pre-training dataset to obtain a trained encoder includes:

[0021] Inputting the pre-training dataset into the encoder pre-training module, and respectively inputting the fruit surface picture and the fruit vein picture in the image pair into the encoder to obtain a first encoded vector and a second encoded vector;

[0022] The first encoded vector and the second encoded vector are respectively input into the information extractor to obtain a first embedded vector and a second embedded vector;

[0023] The first embedded vector and the second embedded vector are input into the cross-correlation matrix learner to calculate the correlation matrix; the correlation matrix and a preset target diagonal matrix are input into the set total correlation loss function to calculate the total loss function value; when the total loss function value reaches the minimum value, a trained encoder is obtained.

[0024] Preferably, the total correlation loss function includes:

[0025] L = L uni + L com

[0026]

[0027] Where L represents the total correlation loss function, L uni represents the vein information uniqueness loss function, L com represents the vein information similarity loss function, C cii represents the elements on the diagonal of the similar block of the cross-correlation matrix, C cij represents the elements off the diagonal of the similar block of the cross-correlation matrix, C uii represents the elements on the diagonal of the unique block of the cross-correlation matrix, C uij represents the elements off the diagonal of the unique block of the cross-correlation matrix.

[0028] It should be noted that the pre-training module includes an encoder, an information extractor, and a cross-correlation matrix learner. The encoder is used to extract the latent space representations of the vein information in the fruit surface image and the fruit vein image respectively. The information extractor is used to project the latent space representations into embedding vectors to facilitate better learning by the cross-correlation matrix learner. Using cross-correlation matrix learning can learn information that is not fruit vein information in the fruit image but highly correlated with fruit maturity compared to simply using image semantic segmentation technology. The non-vein information of the fruit still has a certain representation of fruit maturity, while simply using semantic segmentation technology will result in only using the representation of vein information. For the embedding vector, a threshold is set, that is, the front end of the embedding vector is the vein information common to the corresponding images, and the back end is the non-vein information that is highly correlated with the vein information and also has a representation of fruit maturity. This can be trained by calculating the total correlation loss function.

[0029] Preferably, a fruit maturity classification model is constructed based on the trained encoder, including:

[0030] Freeze the network parameters of the trained encoder, and connect a classifier at the output end of the trained encoder to obtain a fruit maturity classification model;

[0031] The classifier includes a fourth fully connected layer, a third batch normalization layer, a third activation function layer, a fifth fully connected layer, a fourth batch normalization layer, a fourth activation function layer, a classification fully connected layer, and a fifth activation function layer connected in sequence.

[0032] It should be noted that when constructing the fruit maturity classification model, the trained encoder in the pre-training module is used, and its network parameters are frozen and do not participate in the optimization of the fruit maturity classification model. The classification fully connected layer of the classifier sets three levels of maturity, including immature, mature, and overripe, and outputs the maturity probability distribution through the fifth activation function layer.

[0033] Preferably, the fruit maturity classification model is trained using the several fruit surface images to obtain a trained fruit maturity classification model, including:

[0034] Input the several fruit surface images into the fruit maturity classification model, and pass through the trained encoder and classifier in sequence to obtain the predicted maturity of each fruit surface image;

[0035] Based on the true maturity and the predicted maturity of each fruit surface image, establish a cross-entropy loss function;

[0036] Calculate the function value of the cross-entropy loss function and backpropagate to update the network parameters of the classifier;

[0037] When the function value of the cross-entropy loss function reaches the minimum, save the network parameters of the corresponding classifier to obtain a trained classifier;

[0038] Based on the trained encoder and the trained classifier, a trained fruit maturity classification model is formed.

[0039] Preferably, the cross-entropy loss function includes:

[0040]

[0041] where L loss represents the cross-entropy loss function, N represents the number of fruit surface pictures, M represents the number of maturity categories, c represents the c-th maturity, i represents the i-th fruit surface picture, and y ic represents whether the fruit corresponding to the i-th fruit surface picture is the c-th maturity. If so, the value is 1; otherwise, it is 0. p ic represents the probability that the fruit corresponding to the i-th fruit surface picture is predicted to be the c-th maturity.

[0042] The present invention also provides a fruit maturity classification system based on vein information for implementing the above-mentioned fruit maturity classification method based on vein information, including:

[0043] A data acquisition module that acquires a plurality of fruit surface pictures of fruits with different maturities and performs masking processing on each of the fruit surface pictures to obtain corresponding fruit vein pictures;

[0044] A pre-training dataset construction module for forming image pairs from the corresponding fruit surface pictures and fruit vein pictures to construct a pre-training dataset;

[0045] An encoder pre-training module for constructing an encoder pre-training module, including an encoder, an information extractor, and a cross-correlation matrix learner; training the encoder pre-training module using the pre-training dataset to obtain a trained encoder;

[0046] A fruit maturity classification model training module that constructs a fruit maturity classification model based on the trained encoder and trains the fruit maturity classification model using the plurality of fruit surface pictures to obtain a trained fruit maturity classification model;

[0047] A maturity classification module for obtaining the surface picture of the fruit to be classified and inputting it into the trained fruit maturity classification model to obtain the maturity category of the fruit to be classified

[0048] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0049] The present invention provides a method for classifying the maturity of fruits based on vein information. First, a number of fruit surface images of fruits with different maturities are obtained, and each fruit surface image is masked to obtain the corresponding fruit vein image. The corresponding fruit surface image and fruit vein image are combined into an image pair to construct a pre-training dataset, so as to learn the information in the fruit surface image that is not fruit vein information but highly related to the fruit maturity. Then, an encoder pre-training module is constructed, including an encoder, an information extractor, and a cross-correlation matrix learner. The encoder pre-training module is trained using the pre-training dataset to obtain a trained encoder. A fruit maturity classification model is constructed based on the trained encoder, and the fruit maturity classification model is trained using a number of fruit surface images to obtain a trained fruit maturity classification model. Finally, the surface image of the fruit to be classified is obtained and input into the trained fruit maturity classification model to obtain the maturity category of the fruit to be classified. The present invention can effectively utilize the vein information in the fruit surface image, realize accurate identification and classification of the fruit maturity, and has high robustness and strong generalization ability. Description of the Drawings

[0050] Figure 1 It is a flowchart of a method for classifying the maturity of fruits based on vein information according to Embodiment 1;

[0051] Figure 2 It is a schematic structural diagram of the encoder according to Embodiment 2;

[0052] Figure 3 It is a schematic structural diagram of the first-stage unit, second-stage unit, third-stage unit, and fourth-stage unit according to Embodiment 2;

[0053] Figure 4 It is a schematic structural diagram of the first residual block, fourth residual block, eighth residual block, and fourteenth residual block according to Embodiment 2;

[0054] Figure 5 It is a schematic structural diagram of the second residual block, third residual block, fifth residual block, sixth residual block, seventh residual block, ninth residual block, tenth residual block, eleventh residual block, twelfth residual block, thirteenth residual block, fifteenth residual block, and sixteenth residual block according to Embodiment 2;

[0055] Figure 6 It is a schematic structural diagram of the information extractor according to Embodiment 2;

[0056] Figure 7 It is a schematic diagram of training the encoder pre-training module according to Embodiment 2;

[0057] Figure 8 It is a schematic structural diagram of the fruit maturity classification model according to Embodiment 2;

[0058] Figure 9 It is a schematic structural diagram of a fruit maturity classification system based on vein information described in Embodiment 3. Detailed implementation manners

[0059] The accompanying drawings are only for illustrative purposes and should not be construed as limitations on this patent.

[0060] For better illustration of this embodiment, some components in the accompanying drawings are omitted, enlarged or reduced, which do not represent the dimensions of the actual product.

[0061] For those skilled in the art, it is understandable that some well-known structures and their descriptions in the accompanying drawings may be omitted.

[0062] The technical solutions of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0063] Embodiment 1

[0064] This embodiment provides a method for classifying fruit maturity based on vein information, as Figure 1 shown, including:

[0065] S1: Obtain a number of fruit surface pictures of fruits with different maturities, and perform masking processing on each of the fruit surface pictures to obtain corresponding fruit vein pictures;

[0066] S2: Combine the corresponding fruit surface pictures and fruit vein pictures into image pairs to construct a pre-training data set;

[0067] S3: Construct an encoder pre-training module, including an encoder, an information extractor, and a cross-correlation matrix learner; use the pre-training data set to train the encoder pre-training module to obtain a trained encoder;

[0068] S4: Construct a fruit maturity classification model based on the trained encoder, and use the number of fruit surface pictures to train the fruit maturity classification model to obtain a trained fruit maturity classification model;

[0069] S5: Obtain the surface picture of the fruit to be classified, input it into the trained fruit maturity classification model, and obtain the maturity category of the fruit to be classified

[0070] In the specific implementation process, in this embodiment, several fruit surface images of fruits with different maturities are first obtained, and each fruit surface image is masked to obtain the corresponding fruit vein image. The corresponding fruit surface image and fruit vein image are combined into an image pair to construct a pre-training dataset, so as to learn the information in the fruit surface image that is not fruit vein information but highly related to the fruit maturity; then an encoder pre-training module is constructed, including an encoder, an information extractor, and a cross-correlation matrix learner; the encoder pre-training module is trained using the pre-training dataset to obtain a trained encoder; a fruit maturity classification model is constructed based on the trained encoder, and several fruit surface images are used to train the fruit maturity classification model to obtain a trained fruit maturity classification model; finally, the surface image of the fruit to be classified is obtained and input into the trained fruit maturity classification model to obtain the maturity category of the fruit to be classified. The present invention can effectively utilize the vein information in the fruit surface image to achieve accurate recognition and classification of the fruit maturity, and has high robustness and strong generalization ability.

[0071] Embodiment 2

[0072] This embodiment provides a method for classifying fruit maturity based on vein information, including:

[0073] S1: Obtain several fruit surface images of fruits with different maturities, and perform masking processing on each of the fruit surface images to obtain the corresponding fruit vein images;

[0074] It should be noted that the veins presented on the fruit surface are an important part of the fruit structure, and their distribution, color, thickness and other characteristics are closely related to the maturity. The veins of mature fruits are more evenly distributed, while the veins of immature fruits are relatively concentrated; the veins of mature fruits are darker in color, while the veins of immature fruits are lighter in color; the veins of mature fruits are thicker, while the veins of immature fruits are thinner. The selected fruit varieties with different maturities are fruits with obvious vein characteristics, and different maturities include immature, mature and over-ripe. This vein information maps the characteristics of different maturity stages of the fruit. By extracting and analyzing the fruit vein information as an important feature, accurate recognition and classification of the fruit maturity can be achieved.

[0075] Before constructing the pre-training dataset, it is also necessary to preprocess the fruit surface images and fruit vein images, including any one or more of adjusting the resolution, normalizing, cropping or padding to adapt to the model input size.

[0076] S2: Combine the corresponding fruit surface image and fruit vein image into an image pair to construct a pre-training dataset;

[0077] S3: Construct an encoder pre-training module, including an encoder, an information extractor, and a cross-correlation matrix learner; use the pre-training dataset to train the encoder pre-training module to obtain a trained encoder;

[0078] It should be noted that the pre-training module includes an encoder, an information extractor, and a cross-correlation matrix learner. The encoder is used to extract the latent space representations of the vein information in the fruit surface image and the fruit vein image respectively. The information extractor is used to project the latent space representation into an embedding vector to facilitate better learning by the cross-correlation matrix learner. Among them, as Figure 2 shown, the encoder includes an initial convolutional layer, a max pooling layer, a first-stage unit, a second-stage unit, a third-stage unit, a fourth-stage unit, and an average pooling layer connected in sequence;

[0079] As Figure 3 shown, the first-stage unit includes a first residual block, a second residual block, and a third residual block connected in sequence;

[0080] The second-stage unit includes a fourth residual block, a fifth residual block, a sixth residual block, and a seventh residual block connected in sequence;

[0081] The third-stage unit includes an eighth residual block, a ninth residual block, a tenth residual block, an eleventh residual block, a twelfth residual block, and a thirteenth residual block connected in sequence;

[0082] The fourth-stage unit includes a fourteenth residual block, a fifteenth residual block, and a sixteenth residual block connected in sequence.

[0083] Preferably, the information extractor includes a first fully connected layer, a first batch normalization layer, a first activation function layer, a second fully connected layer, a second batch normalization layer, a second activation function layer, and a third fully connected layer connected in sequence.

[0084] Among them, as Figure 4 shown, the structures of the first residual block, the fourth residual block, the eighth residual block, and the fourteenth residual block are the same, and each includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a residual connection convolutional layer, a first residual connector, and a sixth activation function layer;

[0085] The first convolutional layer, the second convolutional layer, the third convolutional layer, the first residual connector, and the sixth activation function layer are connected in sequence;

[0086] The input end of the first convolutional layer is also connected to the input end of the residual connection convolutional layer, and the output end of the residual connection convolutional layer is also connected to the input end of the first residual connector.

[0087] As Figure 5As shown, the structures of the second residual block, the third residual block, the fifth residual block, the sixth residual block, the seventh residual block, the ninth residual block, the tenth residual block, the eleventh residual block, the twelfth residual block, the thirteenth residual block, the fifteenth residual block, and the sixteenth residual block are the same, and each includes a fourth convolutional layer, a fifth convolutional layer, a sixth convolutional layer, a second residual connector, and a seventh activation function layer connected in sequence;

[0088] The input end of the fourth convolutional layer is also connected to the input end of the second residual connector.

[0089] As Figure 6 shown, the information extractor includes a first fully connected layer, a first batch normalization layer, a first activation function layer, a second fully connected layer, a second batch normalization layer, a second activation function layer, and a third fully connected layer connected in sequence.

[0090] As Figure 7 shown, training the encoder pre-training module using the pre-training dataset to obtain a trained encoder includes:

[0091] Inputting the pre-training dataset into the encoder pre-training module, and respectively inputting the fruit surface picture and the fruit vein picture in the image pair into the encoder to obtain a first coding vector and a second coding vector;

[0092] The first coding vector and the second coding vector are respectively input into the information extractor to obtain a first embedding vector and a second embedding vector;

[0093] The first embedding vector and the second embedding vector are input into the cross-correlation matrix learner to calculate the correlation matrix;

[0094] Inputting the correlation matrix and the preset target diagonal matrix into the set total correlation loss function to calculate the total loss function value; when the total loss function value reaches the minimum, a trained encoder is obtained; where the total correlation loss function includes:

[0095] L = L uni + L com

[0096]

[0097] Among them, L represents the total correlation loss function, L uni represents the vein information uniqueness loss function, L com represents the vein information similarity loss function, C cii represents the elements on the diagonal of the similar block of the cross-correlation matrix, C cij represents the elements on the non-diagonal of the similar block of the cross-correlation matrix, C uii represents the elements on the diagonal of the unique block of the cross-correlation matrix, C uijRepresents the elements on the off - diagonal of the unique block of the cross - correlation matrix.

[0098] It should be noted that compared with simply using image semantic segmentation technology, using cross - correlation matrix learning can learn the information of non - fruit veins in fruit pictures but highly correlated with fruit maturity. The non - vein information of the fruit still has a certain representation of fruit maturity, while simply using semantic segmentation technology will only use the representation of vein information. For the embedding vector, in this embodiment, an 80% threshold is set, that is, the front 80% of the embedding vector is the common vein information of the corresponding pictures, and the back 20% is non - vein information highly correlated with the vein information and also having a representation of fruit maturity. This can be trained by calculating the total correlation loss function.

[0099] S4: Based on the trained encoder, construct a fruit maturity classification model, and use the several fruit surface pictures to train the fruit maturity classification model to obtain a trained fruit maturity classification model;

[0100] Based on the trained encoder, construct a fruit maturity classification model, as Figure 8 shown, including:

[0101] Freeze the network parameters of the trained encoder, and connect a classifier at the output end of the trained encoder to obtain a fruit maturity classification model;

[0102] The classifier includes a fourth fully - connected layer, a third batch normalization layer, a third activation function layer, a fifth fully - connected layer, a fourth batch normalization layer, a fourth activation function layer, a classification fully - connected layer, and a fifth activation function layer connected in sequence.

[0103] Using the several fruit surface pictures to train the fruit maturity classification model to obtain a trained fruit maturity classification model includes:

[0104] Input the several fruit surface pictures into the fruit maturity classification model, and successively pass through the trained encoder and classifier to obtain the predicted maturity of each fruit surface picture;

[0105] Based on the true maturity and predicted maturity of each fruit surface picture, establish a cross - entropy loss function;

[0106] Calculate the function value of the cross - entropy loss function, and back - propagate to update the network parameters of the classifier;

[0107] When the function value of the cross - entropy loss function reaches the minimum, save the corresponding network parameters of the classifier to obtain a trained classifier;

[0108] Based on the trained encoder and the trained classifier, form a trained fruit maturity classification model.

[0109] The cross - entropy loss function includes:

[0110]

[0111] Among them, L loss represents the cross - entropy loss function, N represents the number of fruit surface pictures, M represents the number of maturity categories, c represents the c - th maturity, i represents the i - th fruit surface picture, and y ic represents whether the fruit corresponding to the i - th fruit surface picture is the c - th maturity. If it is, the value is 1; otherwise, it is 0. And p ic represents the probability that the fruit corresponding to the i - th fruit surface picture is predicted to be the c - th maturity.

[0112] S5: Obtain the surface picture of the fruit to be classified, input it into the trained fruit maturity classification model, and obtain the maturity category of the fruit to be classified.

[0113] Embodiment 3

[0114] This embodiment provides a fruit maturity classification system based on vein information for implementing the fruit maturity classification method based on vein information described in Embodiment 1 or 2. As Figure 9 shown, it includes:

[0115] A data acquisition module that acquires several fruit surface pictures of fruits with different maturities, and performs masking processing on each of the fruit surface pictures to obtain corresponding fruit vein pictures;

[0116] A pre - training dataset construction module for forming image pairs from the corresponding fruit surface pictures and fruit vein pictures to construct a pre - training dataset;

[0117] An encoder pre - training module for constructing an encoder pre - training module, including an encoder, an information extractor, and a cross - correlation matrix learner; training the encoder pre - training module with the pre - training dataset to obtain a trained encoder;

[0118] A fruit maturity classification model training module that constructs a fruit maturity classification model based on the trained encoder, and trains the fruit maturity classification model with the several fruit surface pictures to obtain a trained fruit maturity classification model;

[0119] A maturity classification module for obtaining the surface picture of the fruit to be classified, inputting it into the trained fruit maturity classification model, and obtaining the maturity category of the fruit to be classified.

[0120] The same or similar reference numerals correspond to the same or similar components;

[0121] The terms used to describe the positional relationship in the attached drawings are for illustrative purposes only and should not be construed as a limitation of this patent;

[0122] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention and are not intended to limit the implementation manners of the present invention. For those of ordinary skill in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to enumerate all implementation manners here. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the claims of the present invention.

Claims

1. A method for classifying the maturity of fruits based on vein information, characterized in that, Including: S1: Obtain several fruit surface images of fruits with different maturities, and perform masking processing on each of the fruit surface images to obtain corresponding fruit vein images; S2: Combine the corresponding fruit surface images and fruit vein images into image pairs to construct a pre-training dataset; S3: Construct an encoder pre-training module, including an encoder, an information extractor, and a cross-correlation matrix learner; use the pre-training dataset to train the encoder pre-training module to obtain a trained encoder; S4: Based on the trained encoder, construct a fruit maturity classification model, and use the several fruit surface images to train the fruit maturity classification model to obtain a trained fruit maturity classification model; S5: Obtain the surface image of the fruit to be classified, input it into the trained fruit maturity classification model, and obtain the maturity category of the fruit to be classified.

2. The method for classifying fruit maturity based on context information according to claim 1, characterized in that, The encoder includes an initial convolutional layer, a max pooling layer, a first-stage unit, a second-stage unit, a third-stage unit, a fourth-stage unit, and an average pooling layer connected in sequence.

3. The method for classifying the fruit maturity based on context information according to claim 2, wherein The first-stage unit includes a first residual block, a second residual block, and a third residual block connected in sequence; The second-stage unit includes a fourth residual block, a fifth residual block, a sixth residual block, and a seventh residual block connected in sequence; The third-stage unit includes an eighth residual block, a ninth residual block, a tenth residual block, an eleventh residual block, a twelfth residual block, and a thirteenth residual block connected in sequence; The fourth-stage unit includes a fourteenth residual block, a fifteenth residual block, and a sixteenth residual block connected in sequence.

4. The method for classifying the fruit maturity based on context information according to claim 1, characterized in that, The information extractor includes a first fully connected layer, a first batch normalization layer, a first activation function layer, a second fully connected layer, a second batch normalization layer, a second activation function layer, and a third fully connected layer connected in sequence.

5. The method for classifying fruit maturity based on context information according to claim 1, characterized in that, Using the pre-training dataset to train the encoder pre-training module to obtain a trained encoder includes: Input the pre-training dataset into the encoder pre-training module, input the fruit surface image and the fruit vein image in the image pair into the encoder respectively to obtain a first encoded vector and a second encoded vector; Input the first encoded vector and the second encoded vector into the information extractor respectively to obtain a first embedding vector and a second embedding vector; Input the first embedding vector and the second embedding vector into the cross-correlation matrix learner to calculate the correlation matrix; input the correlation matrix and a preset target diagonal matrix into the set total correlation loss function to calculate the total loss function value; when the total loss function value reaches the minimum value, obtain a trained encoder.

6. The method for classifying fruit maturity based on context information according to claim 5, wherein The total correlation loss function includes: L = L uni + L com Among them, L represents the total correlation loss function, L uni represents the vein information uniqueness loss function, L com represents the vein information similarity loss function, C cii represents the elements on the diagonal of the similar block of the cross-correlation matrix, C cij represents the elements off the diagonal of the similar block of the cross-correlation matrix, C uii represents the elements on the diagonal of the unique block of the cross-correlation matrix, C uij represents the elements off the diagonal of the unique block of the cross-correlation matrix.

7. The method for classifying the maturity of fruits based on context information according to claim 1, wherein Based on the trained encoder to construct a fruit maturity classification model, including: Freeze the network parameters of the trained encoder, and connect a classifier at the output end of the trained encoder to obtain a fruit maturity classification model; The classifier includes a fourth fully connected layer, a third batch normalization layer, a third activation function layer, a fifth fully connected layer, a fourth batch normalization layer, a fourth activation function layer, a classification fully connected layer, and a fifth activation function layer connected in sequence.

8. The method for classifying fruit maturity based on context information according to claim 7, characterized in that, Training the fruit maturity classification model using the several fruit surface pictures to obtain a trained fruit maturity classification model, including: Inputting the several fruit surface pictures into the fruit maturity classification model, and sequentially passing through the trained encoder and classifier to obtain the predicted maturity of each fruit surface picture; Based on the true maturity and predicted maturity of each fruit surface picture, establishing a cross-entropy loss function; Calculating the function value of the cross-entropy loss function and backpropagating to update the network parameters of the classifier; When the function value of the cross-entropy loss function reaches the minimum, saving the network parameters of the corresponding classifier to obtain a trained classifier; Based on the trained encoder and the trained classifier, forming a trained fruit maturity classification model.

9. The method for classifying fruit maturity based on context information according to claim 8, wherein The cross-entropy loss function includes: Among them, L loss represents the cross-entropy loss function, N represents the number of fruit surface images, M represents the number of maturity categories, c represents the c-th maturity, i represents the i-th fruit surface image, and y ic represents whether the fruit corresponding to the i-th fruit surface image is of the c-th maturity. If so, the value is 1; otherwise, it is 0. p ic represents the probability that the fruit corresponding to the i-th fruit surface image is predicted to be of the c-th maturity.

10. A fruit maturity classification system based on vein information, which is used to implement the fruit maturity classification method based on vein information described in claims 1-9, characterized in that, Including: A data acquisition module, which acquires several fruit surface pictures of fruits with different maturities, and performs mask processing on each of the fruit surface pictures to obtain corresponding fruit vein pictures; A pre-training dataset construction module, which is used to form image pairs from the corresponding fruit surface pictures and fruit vein pictures to construct a pre-training dataset; An encoder pre-training module, which is used to construct an encoder pre-training module, including an encoder, an information extractor, and a cross-correlation matrix learner; training the encoder pre-training module using the pre-training dataset to obtain a trained encoder; A fruit maturity classification model training module, which constructs a fruit maturity classification model based on the trained encoder, and trains the fruit maturity classification model using the several fruit surface pictures to obtain a trained fruit maturity classification model; A maturity classification module, which is used to acquire the surface picture of the fruit to be classified, input it into the trained fruit maturity classification model, and obtain the maturity category of the fruit to be classified.

Images