A silage corn harvester feeding amount prediction system and method based on deep learning

Abstract

The application discloses a silage corn harvester feeding amount prediction system and method based on deep learning, and belongs to the technical field of deep learning. The system comprises an image recognition module, an image processing module, a mechanical parameter acquisition module, a model establishment module, a weighing module and a data processing module. The method comprises the following steps: collecting image information; processing the collected image; collecting the walking speed of the harvester and the feeding roller torque and speed information; processing the collected information by using a lightweight deep learning network model to obtain a predicted feeding amount F1; weighing to obtain an actual feeding amount F2; comparing the predicted feeding amount F1 and the actual feeding amount F2 to optimize the feeding amount prediction model; and adjusting the feeding amount in real time according to the prediction result of the optimized feeding amount prediction model. The application can accurately predict the silage corn feeding amount in advance, timely adjust the working state of the harvester, and overcome the defects that the running speed or cutting speed of the harvester does not match the feeding target amount.

Description

Technical Field

[0001] This invention relates to the field of deep learning technology, and more specifically, to a deep learning-based system and method for predicting the feed amount of a silage corn harvester. Background Technology

[0002] Currently, silage corn harvesters are widely used in silage corn harvesting operations. However, the intelligence of domestic silage corn harvesters is insufficient. During harvesting, the operator usually relies on their own experience to estimate the density of the silage corn and adjust the walking speed, feed roller torque, and rotation speed accordingly. However, human estimation can easily lead to problems such as the speed being too fast or too slow. This results in inaccurate prediction of the feed amount and often causes the running speed or cutting speed to be mismatched with the desired target amount, leading to blockages in the subsequent parts and large losses. Summary of the Invention

[0003] The purpose of this invention is to provide a deep learning-based feeding amount prediction system and method for silage corn harvesters, in order to overcome the shortcomings of existing technologies where the feeding amount of silage corn harvesters cannot be accurately predicted during operation, resulting in a mismatch between the operating speed or cutting speed and the target feeding amount.

[0004] This invention is achieved using the following technical solution:

[0005] A deep learning-based feeding rate prediction system for silage corn harvesters includes:

[0006] The image recognition module is used to capture and recognize images of silage corn;

[0007] The image processing module is used to process the images acquired by the image recognition module in preparation for subsequent model training.

[0008] The mechanical parameter acquisition module is used to collect the walking speed, torque and speed of the feed rollers during the operation of the silage corn harvester to help optimize the feed amount prediction model.

[0009] The model building module uses a lightweight deep learning network model combined with the plant height and density information of silage corn obtained from the model calculation to build a feeding amount prediction model.

[0010] The weighing module is used to collect real-time data on the weight changes of the silage corn fed into the silage corn harvester;

[0011] The data processing module compares the model's predicted feed rate with the actual data and feeds it back to the model building module for model optimization. Based on the data from the optimized feed rate prediction model, it adjusts the walking speed of the silage corn harvester and the torque and speed of the feed rollers in a timely manner, thereby regulating the harvester's feed rate and working status.

[0012] Furthermore, the image recognition module includes a depth camera, the bottom of which is connected to the roof of the silage corn harvester by a gimbal with strong magnets.

[0013] Furthermore, the mechanical parameter acquisition module includes a feed roller torque detection device and a travel speed detection device. The feed roller torque detection device includes a speed sensor, a coupler, and a photoelectric sensor. The speed sensor is attached to the strain shaft. The travel speed detection device includes a differential GPS speed sensor and a base station. The differential GPS speed sensor antenna is installed in the middle and rear of the silage corn harvester's roof, respectively.

[0014] A deep learning-based method for predicting the feed rate of a silage corn harvester includes the following steps:

[0015] S1: Acquire RGB and depth images of silage corn through the image recognition module;

[0016] S2: The acquired images are processed through the image processing module;

[0017] S3: Collects information on the walking speed, feed roller torque, and rotation speed of the silage corn harvester through the mechanical parameter acquisition module;

[0018] S4: The model building module uses a lightweight deep learning network model to process the collected information, calculate the plant height and density of silage corn, and combine this with the information collected by the acquisition module to predict the feeding amount of the silage corn harvester, obtaining the predicted feeding amount F1, expressed by the formula:

[0019]

[0020] F represents the feed rate, L represents the cutting width, h represents the plant height of the silage corn, v represents the walking speed, and ρ represents the density of the silage corn.

[0021] S5: The silage corn fed into the harvester is weighed in real time by the weighing module to obtain the actual feeding amount F2;

[0022] S6: The data processing module compares the predicted feed amount F1 with the actual feed amount F2, verifies the accuracy of the predicted feed amount F1 based on the actual feed amount F2, and feeds the comparison information back to the model building module, which then optimizes the feed amount prediction model in a timely manner.

[0023] S7: The data processing module adjusts the walking speed, feed roller torque, and rotation speed based on the prediction results of the optimized feed amount prediction model, thereby enabling real-time adjustment of the feed amount in advance.

[0024] Furthermore, the method for processing the acquired image in step S2 includes:

[0025] A target density recognition model is established based on a lightweight deep learning network model. The sample images are divided into six levels: low density, low density, medium density, high density, high density, and extra-high density. Data augmentation is performed by rotating the angle, adjusting the brightness, adding noise, and color jittering to increase the number of sample images.

[0026] The obtained sample images were divided into training and test sets in a 9:1 ratio. The test set was then transformed six times using Gaussian blur to improve the model's generalization ability. Gaussian blur uses a normal distribution to calculate the transformation of each pixel in the image. The specific two-dimensional normal distribution equation is as follows:

[0027]

[0028]

[0029] u and v represent the horizontal and vertical distances from the starting pixel, respectively, and r is the blur radius. The standard deviation of a normal distribution.

[0030] Furthermore, in step S4, the lightweight deep learning network model uses a five-point sampling method to measure the plant height and density information of silage corn based on the collected image information.

[0031] Furthermore, the lightweight deep learning network model includes the MobileDenseNet network and the DRSAN network.

[0032] Furthermore, the MobileDenseNet network adds concatenation to the first neck, using bottleneck reduction to decrease computation, and uses depthwise separable convolutions to create a feature connectivity pyramid network, expressed as follows:

[0033]

[0034] The backbone outputs are C3, C4, C5, C6, and C7, while the cervical outputs are P3, P4, P5, P6, and P7. This represents pointwise convolution. Indicates the upsampling function;

[0035] All five feature maps from p3 to p7 use a filter of size 256, selected in Half-Share mode, and the header is represented as follows:

[0036]

[0037] P3, P4, P5, P6, and P7 are the neck outputs, while H1, H2, and H3 are three different head modules with independent weights.

[0038] Furthermore, the DRSAN network includes: a feature extraction module, a nonlinear mapping module, and a reconstruction module.

[0039] The feature extraction module feeds the input LR image into the first convolutional layer, with the following expression:

[0040]

[0041] This indicates a feature extraction operation. Indicates output features, The input image is represented; the first convolutional layer performs shallow feature extraction and sends the output features to the nonlinear mapping module;

[0042] The nonlinear mapping module extracts drag information to reconstruct the HR image and processes shallow features. The expression is as follows:

[0043]

[0044] K represents the number of DRAGs in DRSAN. This represents the Kth DRAG operation;

[0045] Given DRAG input features, the DRA parameters are determined using DRM, expressed as follows:

[0046]

[0047]

[0048]

[0049] In the formula, This indicates dynamic remaining module operations. Indicates global average pooling. Indicates a convolutional layer. The input characteristics of DRAG are represented by N, and the number of residual blocks in one DRAG operation is represented by N.

[0050] DRM learns the correlation between the input image and the residual path, helping the network to mine suitable features from different combinations of features. The residual feature expression for the Nth residual block is:

[0051]

[0052] This represents the convolution operation of the Nth residual block. Let α represent the dynamic residual characteristics of the Nth residual block, and let α represent the residual self-attention coefficient.

[0053] Dynamic residual features It can be represented as:

[0054]

[0055] Reconstruction module, extracting deep features ,pass The r network is upgraded, expressed as:

[0056]

[0057] In the expression For upsampling function, the It consists of a convolutional layer and a shuffle layer. Before the shuffle operation, the... Add to In the process, the pixel-shuffle layer transforms the shape of the input features, and the upgraded mesh reconstructs the SR image to obtain the ISR image, expressed as:

[0058] .

[0059] Compared with the prior art, the beneficial effects of the present invention are:

[0060] (1) This invention compares the predicted feed amount F1 with the actual feed amount F2, verifies the accuracy of the predicted feed amount F1 based on the actual feed amount F2, and feeds the comparison information back to the model building module. The model building module optimizes and trains the feed amount prediction model in a timely manner. The trained model is combined with the collected feed roller torque, speed and walking speed information to predict the density level of silage corn, thereby predicting the silage corn feed amount of the harvester. Based on the prediction results, the harvester feed roller torque, speed and walking speed are automatically adjusted to optimize the silage corn feed amount of the harvester. This can effectively improve the accuracy of silage corn harvest feed amount prediction and improve harvesting efficiency.

[0061] (2) The present invention also designs and improves a lightweight deep learning network model, improves the parameter calculation method, and improves the training speed of the model. Attached Figure Description

[0062] Figure 1 This is a flowchart illustrating a deep learning-based feeding amount prediction system for silage corn harvesters provided by the present invention.

[0063] Figure 2 This is a flowchart illustrating a deep learning-based method for predicting the feed amount of a silage corn harvester, as provided by this invention.

[0064] Figure 3 This is a schematic diagram of the lightweight deep learning network model provided by the present invention. Detailed Implementation

[0065] The present invention will now be described in conjunction with the accompanying drawings and specific embodiments.

[0066] Example 1: See Figure 1 As shown in this embodiment, a deep learning-based feeding prediction system for silage corn harvesters includes an image recognition module, an image processing module, a mechanical parameter acquisition module, a model building module, a weighing module, and a data processing module.

[0067] The image recognition module includes a depth camera, which is attached to the roof of the silage corn harvester by a gimbal with strong magnets at the bottom, for capturing and recognizing images of silage corn.

[0068] The image processing module is used to process the images acquired by the image recognition module in preparation for subsequent model training.

[0069] The mechanical parameter acquisition module includes a feed roller speed detection device and a travel speed detection device. The feed roller speed detection device includes a speed sensor, a coupler, and a photoelectric sensor. It analyzes the resistance changes of strain gauges and designs a detection circuit. The feed roller speed detection device is attached to the strain gauge shaft. The travel speed detection device includes a differential GPS speed sensor and a base station. The differential GPS speed sensor antennas are installed in the middle and rear of the silage corn harvester's roof, respectively. Based on the data from the mobile station, the coordinates of the mobile station are calculated and converted into travel speed. The mechanical parameter acquisition module is used to collect the travel speed, torque, and speed of the feed roller during the operation of the silage corn harvester to help optimize the feed amount prediction model.

[0070] The model building module uses a lightweight deep learning network model combined with the plant height and density information of silage corn obtained from model calculations to build a feeding amount prediction model.

[0071] The weighing module, located inside the storage silo, is used to collect real-time data on the weight changes of the silage corn fed into the silage corn harvester.

[0072] The data processing module compares the model's predicted feed rate with the actual data and feeds it back to the model building module for model optimization. Based on the data from the optimized feed rate prediction model, it adjusts the walking speed of the silage corn harvester and the torque and speed of the feed rollers in a timely manner, thereby regulating the harvester's feed rate and working status.

[0073] Example 2: See Figure 2 , Figure 3As shown, this embodiment provides a deep learning-based method for predicting the feed amount of a silage corn harvester, including the following steps:

[0074] S1: Acquire RGB and depth images of silage corn using the depth camera in the image recognition module;

[0075] S2: The acquired images are processed through the image processing module;

[0076] S3: Obtain information on the torque, speed, and travel speed of the feed roller of the silage corn harvester through the mechanical parameter acquisition module;

[0077] S4: The model building module uses a lightweight deep learning network model to process the data from the image recognition module, calculating the plant height and density of silage corn. Combined with the walking speed and feed roller torque and speed information collected by the acquisition module, the feed rate of the silage corn harvester is predicted to obtain the predicted feed rate F1, expressed by the formula:

[0078]

[0079] F represents the feed rate, L represents the cutting width, h represents the plant height of the silage corn, v represents the walking speed, and ρ represents the density of the silage corn.

[0080] S5: The silage corn fed into the harvester is weighed in real time by the weighing module to obtain the actual feeding amount F2;

[0081] S6: The data processing module compares the predicted feed amount F1 with the actual feed amount F2, verifies the accuracy of the feed amount F1 based on the feed amount F2, and feeds the comparison information back to the model building module, which then optimizes the feed amount prediction model in a timely manner.

[0082] S7: The data processing module adjusts the walking speed, feed roller torque, and rotation speed based on the prediction results of the optimized feed amount prediction model, thereby enabling real-time adjustment of the feed amount in advance.

[0083] In this embodiment, to obtain the feeding amount data of the silage corn harvester, the density level of the silage corn in the acquired image is obtained and reflected in the subsequent calculation. The density calculated by the model is extracted, and combined with the measured walking speed and feed roller torque and speed information, the predicted feeding amount value is calculated according to the feeding amount prediction formula. The actual feeding amount value obtained by weighing is compared with the actual feeding amount value to optimize the model calculation and obtain a more accurate feeding amount prediction value. Furthermore, through model calculation, the walking speed, feed roller torque, and speed of the harvester are adjusted according to the prediction results of the optimized feeding amount prediction model, thereby adjusting the harvester to maintain a suitable feeding amount and matching working state, solving the problem of mismatch between its running speed or cutting speed and the desired target amount, and preventing machine blockage.

[0084] Specifically, the method for processing the obtained image in step S2 includes:

[0085] A target density recognition model is established based on a lightweight deep learning neural network. The sample images are divided into six levels: low density, low density, medium density, high density, high density, and extra-high density. Data augmentation is performed by rotating the image, adjusting the brightness, adding noise, and color jittering to increase the number of sample images.

[0086] The obtained sample images were divided into training and test sets in a 9:1 ratio. The test set was then transformed six times using Gaussian blur to improve the model's generalization ability. Gaussian blur uses a normal distribution to calculate the transformation of each pixel in the image. The specific two-dimensional normal distribution equation is as follows:

[0087]

[0088]

[0089] u and v represent the horizontal and vertical distances from the starting pixel, respectively, and r is the blur radius. The standard deviation of a normal distribution.

[0090] Specifically, in step S4, the lightweight deep learning network model uses a five-point sampling method to measure the plant height and density information of silage corn based on the collected image information.

[0091] The lightweight deep learning network models in this embodiment include the MobileDenseNet network and the DRSAN network.

[0092] Specifically, the MobileDenseNet network adds concatenation to the first neck, using bottlenecks to reduce computation, and uses depthwise separable convolutions to create a feature connection pyramid network, expressed by the formula:

[0093]

[0094] The backbone outputs are C3, C4, C5, C6, and C7, while the cervical outputs are P3, P4, P5, P6, and P7. This represents pointwise convolution. Indicates the upsampling function;

[0095] All five feature maps from p3 to p7 use a filter of size 256, selected in Half-Share mode, and the header is represented as follows:

[0096]

[0097] P3, P4, P5, P6, and P7 are the neck outputs, while H1, H2, and H3 are three different head modules with independent weights.

[0098] Specifically, this embodiment adds DRSAN to the MobileDenseNet network to build a lightweight network model. The DRSAN network includes: a feature extraction module, a nonlinear mapping module, and a reconstruction module.

[0099] The feature extraction module feeds the input LR image into the first convolutional layer, with the following expression:

[0100]

[0101] This indicates a feature extraction operation. Indicates output features, The input image is represented; the first convolutional layer performs shallow feature extraction and sends the output features to the nonlinear mapping module;

[0102] The nonlinear mapping module extracts drag information to reconstruct the HR image and processes shallow features. The expression is as follows:

[0103]

[0104] K represents the number of DRAGs in DRSAN. This represents the Kth DRAG operation;

[0105] Given DRAG input features, the DRA parameters are determined using DRM, expressed as follows:

[0106]

[0107] In the formula, This indicates dynamic remaining module operations. Indicates global average pooling. Indicates a convolutional layer. The input characteristics of DRAG are represented by N, and the number of residual blocks in one DRAG operation is represented by N.

[0108] DRM learns the correlation between the input image and the residual path, helping the network to mine suitable features from different combinations of features. The residual feature expression for the Nth residual block is:

[0109]

[0110] This represents the convolution operation of the Nth residual block. Let α represent the dynamic residual characteristics of the Nth residual block, and let α represent the residual self-attention coefficient.

[0111] Dynamic residual features It can be represented as:

[0112]

[0113] Reconstruction module, extracting deep features ,pass The r network is upgraded, expressed as:

[0114]

[0115] In the expression For upsampling function, the It consists of a convolutional layer and a shuffle layer. Before the shuffle operation, the... Add to In the process, the pixel-shuffle layer transforms the shape of the input features, and the upgraded mesh reconstructs the SR image to obtain the ISR image, expressed as:

[0116] .

[0117] The above description is merely a specific embodiment of this application. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the protection scope of this application. The protection scope of this application should be determined by the protection scope of the claims.

Claims

1. A deep learning-based feeding rate prediction system for silage corn harvesters, characterized in that: include: The image recognition module is used to capture and recognize images of silage corn; The image processing module is used to process the images acquired by the image recognition module in preparation for subsequent model training. The mechanical parameter acquisition module is used to collect the walking speed, torque and speed of the feed rollers during the operation of the silage corn harvester to help optimize the feed amount prediction model. The model building module uses a lightweight deep learning network model combined with the plant height and density information of silage corn obtained from the model calculation to build a feeding amount prediction model. The weighing module is used to collect real-time data on the weight changes of the silage corn fed into the silage corn harvester; The data processing module is used to compare the model's feed rate prediction data with the actual data and feed it back to the model building module to optimize the model. Based on the data from the optimized feed rate prediction model, the module adjusts the walking speed of the silage corn harvester and the torque and speed of the feed roller in a timely manner, thereby regulating the harvester's feed rate and working status. The lightweight deep learning network model includes the MobileDenseNet network and the DRSAN network; The MobileDenseNet network adds concatenation at the first neckline to reduce computation by using a bottleneck, and creates a feature connection pyramid network using depthwise separable convolutions, which can be expressed as follows: ； The backbone outputs are C3, C4, C5, C6, and C7, while the cervical outputs are P3, P4, P5, P6, and P7. This represents pointwise convolution. Indicates the upsampling function; All five feature maps from p3 to p7 use a filter of size 256, selected in Half-Share mode, and the header is represented as follows: ； P3, P4, P5, P6, and P7 are the neck outputs, while H1, H2, and H3 are three different head modules with independent weights.

2. The deep learning-based feeding rate prediction system for silage corn harvesters according to claim 1, characterized in that: The image recognition module includes a depth camera, the bottom of which is connected to the roof of the silage corn harvester by a gimbal with strong magnets.

3. The deep learning-based feeding rate prediction system for silage corn harvesters according to claim 1, characterized in that: The mechanical parameter acquisition module includes a feed roller torque detection device and a travel speed detection device. The feed roller torque detection device includes a speed sensor, a coupler, and a photoelectric sensor. The speed sensor is attached to the strain shaft. The travel speed detection device includes a differential GPS speed sensor and a base station. The differential GPS speed sensor antennas are respectively installed in the middle and rear of the silage corn harvester's roof.

4. The deep learning-based feeding rate prediction system for silage corn harvesters according to claim 1, characterized in that, The DRSAN network includes: a feature extraction module, a nonlinear mapping module, and a reconstruction module. The feature extraction module feeds the input LR image into the first convolutional layer, with the following expression: ； This indicates a feature extraction operation. Indicates output features, The input image is represented; the first convolutional layer performs shallow feature extraction and sends the output features to the nonlinear mapping module; The nonlinear mapping module extracts drag information to reconstruct the HR image and processes shallow features. The expression is as follows: ； K represents the number of DRAGs in DRSAN. This represents the Kth DRAG operation; Given DRAG input features, the DRA parameters are determined using DRM, expressed as follows: ; In the formula, This indicates dynamic remaining module operations. Indicates global average pooling. Indicates a convolutional layer. The input features of the DRAG are represented by N, which represents the number of residual blocks in a single DRAG operation. DRM learns the correlation between the input image and the residual path, helping the network to mine suitable features from different combinations of features. The residual feature expression for the Nth residual block is: ； This represents the convolution operation of the Nth residual block. The dynamic residual characteristics of the Nth residual block are represented by α, where α represents the residual self-attention coefficient. Dynamic residual features It can be represented as: ; Reconstruction module, extracting deep features ,pass The r network is upgraded, expressed as: ; In the expression For upsampling function, the It consists of a convolutional layer and a shuffle layer. Before the shuffle operation, the... Add to In the process, the pixel-shuffle layer transforms the shape of the input features, and the upgraded mesh reconstructs the SR image to obtain the ISR image, expressed as: 。 5. A deep learning-based method for predicting the feed rate of a silage corn harvester, implemented based on the deep learning-based silage corn harvester feed rate prediction system described in any one of claims 1-4, characterized in that, Includes the following steps: S1: Acquire RGB and depth images of silage corn through the image recognition module; S2: The acquired images are processed through the image processing module; S3: Collects information on the walking speed, feed roller torque, and rotation speed of the silage corn harvester through the mechanical parameter acquisition module; S4: The model building module uses a lightweight deep learning network model to process the collected information, calculate the plant height and density of silage corn, and combine this with the information collected by the acquisition module to predict the feeding amount of the silage corn harvester, obtaining the predicted feeding amount F1, expressed by the formula: ; F represents the feed rate, L represents the cutting width, h represents the plant height of the silage corn, v represents the walking speed, and ρ represents the density of the silage corn. S5: The silage corn fed into the harvester is weighed in real time by the weighing module to obtain the actual feeding amount F2; S6: The data processing module compares the predicted feed amount F1 with the actual feed amount F2, verifies the accuracy of the predicted feed amount F1 based on the actual feed amount F2, and feeds the comparison information back to the model building module, which then optimizes the feed amount prediction model in a timely manner. S7: The data processing module adjusts the walking speed, feed roller torque, and rotation speed based on the prediction results of the optimized feed amount prediction model, thereby enabling real-time adjustment of the feed amount in advance.

6. The method for predicting the feed amount of a silage corn harvester according to claim 5, characterized in that, The method for processing the acquired image in step S2 includes: A target density recognition model is established based on a lightweight deep learning network model. The sample images are divided into six levels: low density, low density, medium density, high density, high density, and extra-high density. Data augmentation is performed by rotating the angle, adjusting the brightness, adding noise, and color jittering to increase the number of sample images. The obtained sample images were divided into training and test sets in a 9:1 ratio. The test set was then transformed six times using Gaussian blur to improve the model's generalization ability. Gaussian blur uses a normal distribution to calculate the transformation of each pixel in the image. The specific two-dimensional normal distribution equation is as follows: ; ; u and v represent the horizontal and vertical distances from the starting pixel, respectively, and r is the blur radius. The standard deviation of a normal distribution.

7. The method for predicting the feed amount of a silage corn harvester according to claim 5, characterized in that, In step S4, the lightweight deep learning network model uses a five-point sampling method to measure the plant height and density information of silage corn based on the collected image information.

Images