Rapeseed protein and grease content online detection method, device and system based on lightweight model

Abstract

The invention discloses a lightweight model-based rapeseed protein and grease content online detection method, device and system, and belongs to the technical field of analysis and detection. A data set covering images of rapeseeds in Zhong China, South China and East China is constructed, and a rapeseed protein and grease content prediction model is developed based on a convolutional neural network. According to the method, a FasterNet model is selected and used for predicting the protein content and the grease content of rapeseeds, and the FasterNet model is deployed in a mobile terminal. According to the method, nondestructive and rapid prediction of the protein and the oil content of the rapeseed is realized, the mobile terminal supports the user to upload the rapeseed image anytime and anywhere, and the prediction result is displayed in a visual manner, so that real-time prediction and evaluation of the protein and the oil content of the rapeseed are realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of analysis and detection, and more specifically, relates to a method, device and system for online detection of rapeseed protein and oil content based on computer vision. Background Art

[0002] Rapeseed is an important oil crop in the world with extremely high economic and agricultural value. Among them, protein and oil content are core indicators for evaluating its quality and economic benefits. The high oil content improves the efficiency of producing rapeseed oil, which is the main source of healthy, unsaturated fats for human consumption. It also makes it an important raw material for producing biofuels and supports the transition to renewable energy. At the same time, protein-rich rapeseed meal is a by-product of oil extraction and can be used as a high-quality protein source for animal feed, contributing to livestock nutrition. These characteristics make rapeseed not only economically beneficial, but also crucial in addressing food security, sustainable agriculture and energy needs.

[0003] Traditional chemical detection methods for rapeseed protein and oil content, such as Soxhlet extraction and Kjeldahl nitrogen determination, are not only time-consuming and labor-intensive, but also often destructive. They are not suitable for high-throughput analysis and are difficult to meet the needs of modern rapid and non-destructive testing.

[0004] With the rapid development of deep learning, deploying models to local mobile applications has become a hot research area. Due to limited resources, mobile devices have significant challenges in computing power, storage space, and power consumption. Therefore, it is crucial to achieve lightweight and efficient reasoning while ensuring model performance. Summary of the invention

[0005] In view of the above defects or improvement needs of the prior art, the present invention proposes an online detection method, device and system for rapeseed protein and oil content based on a lightweight model, which effectively deploys a deep learning model on a mobile terminal, and achieves a balance between accuracy and reasoning speed through technologies such as format conversion and optimization of the reasoning process, providing a reference for the application of deep learning models in resource-constrained environments.

[0006] To achieve the above object, according to one aspect of the present invention, a method for online detection of rapeseed protein and oil content based on a lightweight model is provided, comprising:

[0007] Establishing a sample data set, wherein the sample data set includes rapeseed images and protein and oil contents corresponding to each rapeseed image;

[0008] Divide the sample data set into training set, validation set and test set;

[0009] Constructing a rapeseed protein and oil content prediction model based on convolutional neural network;

[0010] The prediction model is trained with the training set, tested with the test set, and verified with the validation set.

[0011] In some optional embodiments, the FasterNet model is used to construct a rapeseed protein and oil content prediction model.

[0012] In some optional embodiments, an electron microscope is used in combination with a high-definition CCD industrial camera, a light-proof black box, and a constant-current ring light to capture rapeseed images.

[0013] In some optional implementation schemes, when capturing images, the pixels are set to 680*680 and the resolution is adjusted to 120.

[0014] In some optional implementation schemes, the prediction model is trained based on the Pytorch framework, PytorchMobile is selected as the mobile deployment framework, and the prediction model saved during the training process is converted into a pt format file, and the converted prediction model is embedded in the mobile application to integrate the inference engine.

[0015] According to another aspect of the present invention, there is provided an online detection device for rapeseed protein and oil content based on a lightweight model, comprising:

[0016] A data set establishment module, used to establish a sample data set, wherein the sample data set includes rapeseed images and protein and oil contents corresponding to each rapeseed image;

[0017] A data set partitioning module is used to partition the sample data set into a training set, a validation set, and a test set;

[0018] A model building module, used to build a rapeseed protein and oil content prediction model based on convolutional neural network;

[0019] The training module is used to train the prediction model through the training set, test the prediction model through the test set, and verify the prediction model through the verification set.

[0020] In some optional embodiments, the FasterNet model is used to construct a rapeseed protein and oil content prediction model.

[0021] According to another aspect of the present invention, there is provided an online detection system for rapeseed protein and oil content based on a lightweight model, comprising: a mobile terminal and a camera device;

[0022] The mobile terminal includes any one of the devices described above.

[0023] In some optional embodiments, the photographing device uses an electron microscope, a high-definition CCD industrial camera, and is equipped with a light-proof black box and a constant-current ring light to capture rapeseed images.

[0024] In some optional implementation schemes, when capturing images, the pixels are set to 680*680 and the resolution is adjusted to 120.

[0025] In some optional implementation schemes, the prediction model is trained based on the Pytorch framework, PytorchMobile is selected as the mobile deployment framework, and the prediction model saved during the training process is converted into a pt format file, and the converted prediction model is embedded in the mobile application to integrate the inference engine.

[0026] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects compared with the prior art:

[0027] The present invention effectively deploys deep learning models on mobile terminals, achieves a balance between accuracy and reasoning speed through technologies such as format conversion and optimization of the reasoning process, and provides a reference for applying deep learning models in resource-constrained environments. It also integrates the mobile terminal app and portable camera equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a schematic flow chart of an online detection method for rapeseed protein and oil content based on a lightweight model provided by an embodiment of the present invention;

[0029] Figure 2 is a schematic diagram of a device provided by an embodiment of the present invention;

[0030] Figure 3 The geographical distribution and protein and oil content distribution of rapeseed samples collected in the present invention;

[0031] Figure 4 It is a core structure and training principle diagram of a FasterNet model provided by an embodiment of the present invention;

[0032] Figure 5 It is a core structure diagram of a SwinTransformerV2 model provided in the comparative example of the present invention;

[0033] Figure 6 It is a loss curve diagram of a FasterNet model and a SwinTransformerV2 model provided in the comparative example of the present invention;

[0034] Figure 7It is a fitting curve diagram of the predicted value and the true value of the protein and oil content in the test set of a FasterNet model and a SwinTransformerV2 model provided in the comparative example of the present invention, as well as a sample content distribution diagram;

[0035] Figure 8 It is a schematic diagram of the results of real-time online prediction of rapeseed protein and oil content after a model provided by an embodiment of the present invention is deployed on a mobile terminal. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0037] like Figure 1 FIG. 1 is a flow chart of an online detection method for rapeseed protein and oil content based on a lightweight model provided by an embodiment of the present invention. Figure 1 The method shown includes the following steps:

[0038] S1: Establishing a sample data set, wherein the sample data set includes rapeseed images and protein and oil contents corresponding to each rapeseed image;

[0039] S2: Divide the sample data set into training set, validation set and test set;

[0040] S3: Construction of rapeseed protein and oil content prediction model based on convolutional neural network;

[0041] S4: Train the prediction model through the training set, test the prediction model through the test set, and verify the prediction model through the validation set.

[0042] Furthermore, the determination of protein and oil content in rapeseed can be carried out in the following manner:

[0043] 1) Protein determination: Experimental steps for determining rapeseed protein content using a Kjeldahl nitrogen analyzer:

[0044] 1. Sample weighing: Accurately weigh 0.5 g of rapeseed sample and place it in a clean Kjeldahl flask. Make sure the sample is fully dry to avoid moisture affecting the measurement results.

[0045] 2. Add catalyst and digestion agent: Add 5 grams of potassium sulfate and 0.5 grams of copper sulfate to the sample as catalysts; then carefully add 20 milliliters of concentrated sulfuric acid. Sulfuric acid can accelerate the conversion of organic nitrogen into ammonium nitrogen, and the catalyst improves the reaction efficiency.

[0046] 3. Digestion: Place the flask on the digestion furnace and heat until the sample solution turns transparent and light green (digestion takes about 1-2 hours). During this process, the temperature should be controlled to avoid excessive carbonization of the sample.

[0047] 4. Cooling and dilution: After digestion is complete, remove the flask and cool it. Then carefully transfer the contents to the distillation apparatus and add 50 ml of distilled water for dilution.

[0048] 5. Distillation: Add about 20 ml of 40% sodium hydroxide solution to the distillation device and start distillation immediately. The distilled ammonia is absorbed by a boric acid solution of known concentration. Usually distill for about 10-15 minutes to ensure that the ammonia is completely collected.

[0049] 6. Titration: Titrate the ammonia-absorbed boric acid solution with standard hydrochloric acid solution until the solution shows a color endpoint (usually pink).

[0050] 7. Calculate the protein content: Calculate the nitrogen content of the sample based on the volume of hydrochloric acid consumed, and then multiply it by the protein conversion factor (5.53) to obtain the protein content in rapeseed.

[0051] 2) Determination of crude fat content of rapeseed: Using the Soxhlet extraction method, weigh 5 grams of dried rapeseed sample, put it into a defatted filter paper bag and seal it. Place the filter paper bag in a Soxhlet extractor and add anhydrous ether as a solvent. Heat and reflux the extraction for 6 hours to ensure that the fat is completely dissolved. After the extraction is completed, recover the ether solvent and remove the filter paper bag. Dry the extracted rapeseed sample in an oven at 105°C to constant weight. Record the weight difference of the dried sample and calculate the crude fat content of the rapeseed based on the initial weight.

[0052] Furthermore, labeling of images and data: After adjusting the image, the rapeseed image needs to be labeled to identify the rapeseed in the image.

[0053] Furthermore, the establishment of the model sample data set can be achieved in the following ways:

[0054] A total of 268 rapeseed varieties were included, with 20 images taken for each variety. The dataset was divided into training set, validation set, and test set in a ratio of 8:1:1. Four data augmentation methods were used to process the images in the training set, and finally 21,440 training images were generated. The validation set and test set each contained 536 images. The geographical distribution of the samples and the distribution of rapeseed protein and oil content are shown in the figure below. Figure 3shown.

[0055] Furthermore, for the regression prediction task, the present invention uses commonly used indicators to evaluate the model: mean squared error (MSE), decision coefficient (r-squared, R2), probability of accuracy height 0.85 (probability, PA0.85) and mean accuracy (MA) to evaluate the model performance.

[0056]

[0057] In the formula, n is the number of samples in the validation data set, y i is the true value, is the predicted value, is the predicted mean, C ACC≥0.85 It is the number of samples with a precision above 0.85.

[0058] Table 1 Model parameters and model effect evaluation

[0059]

[0060] Further, Figure 8 This is a graph showing the results of predicting rapeseed protein and oil contents after the model is deployed on the mobile terminal. The protein and oil contents are 42.82% and 25.38% respectively, which are close to the actual contents of 44.38% and 23.22%, and the error is within the allowable range. It takes 6.92 seconds for FasterNet to predict 536 pictures.

[0061] After testing, the determination coefficient (R2) of the FasterNet model in terms of fat and protein content reached 0.9336 and 0.8828, and the root mean square errors were 0.6982 and 0.6498, respectively; the determination coefficient (R2) of the SwinTransformer model in terms of fat and protein content reached 0.9366 and 0.8974, and the RMSE were 0.6822 and 0.6079, respectively. By comparing the model parameter size, FLOPs, and model inference time of SwinTransformer and FasterNet, it was found that FasterNet, as a lightweight model, is more suitable for deploying embedded systems.

[0062] like Figure 4As shown, FasterNet used in the embodiment of the present invention is a new type of neural network backbone network, which was proposed at the CVPR conference in 2023. The core design concept of FasterNet is to reduce the amount of calculation by optimizing the model structure and streamlining the parameters, thereby improving the inference speed of the model. Specific methods include using pruning technology, quantization technology, and distillation learning strategies to reduce the complexity of the network. In addition, FasterNet usually further reduces the amount of calculation by introducing more efficient convolution layer designs, such as depthwise separable convolution or grouped convolution. In the present invention, the code implements the FasterNet model, whose core architecture consists of several key components. The input layer receives a standardized RGB image with an input size of 224×224. The feature extraction module stacks multiple convolutional layers and residual blocks to capture image features. Early convolutional layers extract basic features, while deeper convolutional layers capture more complex global features. The combination of residual blocks and jump connections effectively solves the problem of gradient disappearance in deep networks. After extracting features, the global pooling layer compresses the two-dimensional feature map into a one-dimensional vector. Then, the fully connected layer (classifier) ​​maps this feature vector to the final class label. Finally, the output layer uses the softmax activation function to convert the output of the fully connected layer into a probability distribution and selects the class with the highest probability as the image label. The data is normalized using the mean and standard deviation of the ImageNet dataset. After dividing the dataset into training, validation, and test sets, different augmentation strategies are applied to ensure consistency and improve the performance of the model during training and validation.

[0063] like Figure 5 As shown in the figure, Swin Transformer is a hierarchical Transformer architecture for computer vision tasks, which improves the efficiency and performance of traditional visual Transformers through a sliding window mechanism. This model is suitable for tasks such as image classification, object detection, and semantic segmentation. Its main innovation lies in the sliding window mechanism, which divides the image into multiple small patches and applies a self-attention mechanism in each small patch, thereby significantly reducing the computational complexity. In addition, Swin Transformer adopts a hierarchical architecture and gradually expands the window size to extract features of different scales, which can capture global information and local details at the same time.

[0064] In order to improve the flexibility of the model, Swin Transformer introduces a hierarchical representation, where each layer gradually refines the image features and builds feature representation from local to global. Similar to the hierarchical structure of convolutional neural networks, this method improves the model's migration ability and shows high performance in large-scale image tasks, with an image input size of 256×256.

[0065] Figure 6 It is a loss curve diagram of a FasterNet model and a SwinTransformerV2 model provided in the comparative example of the present invention.

[0066] Figure 7 It is a fitting curve diagram of the predicted value and the true value of the protein and oil content in the test set of a FasterNet model and a SwinTransformerV2 model provided by the comparative example of the present invention, as well as a sample content distribution diagram.

[0067] like Figure 2 As shown, the present invention also provides an online detection device for rapeseed protein and oil content based on a lightweight model, comprising:

[0068] A data set establishment module, used to establish a sample data set, wherein the sample data set includes rapeseed images and protein and oil contents corresponding to each rapeseed image;

[0069] A data set partitioning module is used to partition the sample data set into a training set, a validation set, and a test set;

[0070] A model building module, used to build a rapeseed protein and oil content prediction model based on convolutional neural network;

[0071] The training module is used to train the prediction model through the training set, test the prediction model through the test set, and verify the prediction model through the verification set.

[0072] In some optional implementation schemes, the FasterNet model is used to construct a rapeseed protein and oil content prediction model, and the SwinTransformerV2 model is used for comparative analysis.

[0073] The present application also provides an online detection system for rapeseed protein and oil content based on a lightweight model, comprising: a mobile terminal and a camera device;

[0074] The mobile terminal includes any one of the devices described above.

[0075] In some optional implementation schemes, the camera for collecting rapeseed images uses an electron microscope, a high-definition CCD industrial camera, and a customized light-proof black box and a constant current ring light to ensure the image quality and stability during the collection process. When collecting images, the images are processed, the pixels are set to 680*680, and the resolution is adjusted to 120.

[0076] In some optional implementation schemes, the prediction model is trained based on the Pytorch framework, so Pytorch Mobile is selected as the mobile deployment framework. The saved model during the computer training process is converted into a pt format file, and the converted model is embedded in the mobile application, and the inference engine is integrated to achieve efficient inference of the model. In this process, the configuration of the inference engine needs to be adjusted according to the hardware specifications of the mobile device to improve the execution efficiency of the model on the mobile terminal.

[0077] It should be pointed out that, according to the needs of implementation, the various steps / components described in this application can be split into more steps / components, and two or more steps / components or partial operations of steps / components can be combined into new steps / components to achieve the purpose of the present invention.

[0078] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for online detection of rapeseed protein and oil content based on a lightweight model, characterized in that: include: Establishing a sample data set, wherein the sample data set includes rapeseed images and protein and oil contents corresponding to each rapeseed image; Divide the sample data set into training set, validation set and test set; Constructing a rapeseed protein and oil content prediction model based on convolutional neural network; The prediction model is trained with the training set, tested with the test set, and verified with the validation set.

2. The method according to claim 1, characterized in that The FasterNet model was used to construct a rapeseed protein and oil content prediction model.

3. The method according to claim 1 or 2, characterized in that: Rapeseed images were collected using an electron microscope, a high-definition CCD industrial camera, a light-proof black box, and a constant-current ring light.

4. The method according to claim 3, characterized in that When collecting images, set the pixels to 680*680 and the resolution to 120.

5. The method according to claim 4, characterized in that The prediction model is trained based on the Pytorch framework, Pytorch Mobile is selected as the mobile deployment framework, and the prediction model saved during the training process is converted into a pt format file. The converted prediction model is embedded in the mobile application and the inference engine is integrated.

6. An online detection device for rapeseed protein and oil content based on a lightweight model, characterized in that: include: A data set establishment module, used to establish a sample data set, wherein the sample data set includes rapeseed images and protein and oil contents corresponding to each rapeseed image; A data set partitioning module is used to partition the sample data set into a training set, a validation set, and a test set; A model building module, used to build a rapeseed protein and oil content prediction model based on convolutional neural network; The training module is used to train the prediction model through the training set, test the prediction model through the test set, and verify the prediction model through the verification set.

7. The device according to claim 6, characterized in that The FasterNet model was used to construct a rapeseed protein and oil content prediction model.

8. An online detection system for rapeseed protein and oil content based on a lightweight model, characterized in that: include: Mobile devices and camera equipment; The mobile terminal includes the device described in any one of claims 6 or 7.

9. The system according to claim 8, characterized in that The photographic equipment uses an electron microscope, a high-definition CCD industrial camera, and is equipped with a light-proof black box and a constant-current ring light to collect rapeseed images.

10. The system according to claim 9, characterized in that The prediction model is trained based on the Pytorch framework, Pytorch Mobile is selected as the mobile deployment framework, and the prediction model saved during the training process is converted into a pt format file. The converted prediction model is embedded in the mobile application and the inference engine is integrated.

Images